Quantcast
Viewing all articles
Browse latest Browse all 5648

flume and hadoop append

Exploring the idea of using “append” instead of creating new files with HDFS every few minutes.
Are there particular design decisions / considerations?

There’s certainly a history of append with HDFS, mainly, earlier versions of Hadoop warn strongly against using file append semantics.

-Charles

Viewing all articles
Browse latest Browse all 5648

Trending Articles