Hi
Does flume have support for buffering/staging avro events locally on disk
and storing them in hdfs as parquet files?
Cloudera CDK explains [1] how to do this method manually but ideally I want
this process directly integrated into the flume runtime.
Cheers,
-Kristoffer
1. https://github.com/cloudera/cdk-examples/tree/master/dataset-staging
Does flume have support for buffering/staging avro events locally on disk
and storing them in hdfs as parquet files?
Cloudera CDK explains [1] how to do this method manually but ideally I want
this process directly integrated into the flume runtime.
Cheers,
-Kristoffer
1. https://github.com/cloudera/cdk-examples/tree/master/dataset-staging