Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Parquet buffering capability

$
0
0
Hi

Does flume have support for buffering/staging avro events locally on disk
and storing them in hdfs as parquet files?

Cloudera CDK explains [1] how to do this method manually but ideally I want
this process directly integrated into the flume runtime.

Cheers,
-Kristoffer

1. https://github.com/cloudera/cdk-examples/tree/master/dataset-staging

Viewing all articles
Browse latest Browse all 5648

Trending Articles