Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Ordering of messages in flume-ng

$
0
0
I've seen a fair number of resources on the web that describe the loose
ordering guarantees that flume offers for messages in the face of
degradation or failures. But I can't tell what applies to flume-og, and
flume-ng. Hopefully somebody can help clear up the situation.

In the case of a single agent topology, (source -> FileSystem Channel ->
sink), can messages become out of order? What situations cause that?

In a multi agent topology, does that answer change?

(Agent 1 Source -> FilesystemChannel -> Avro To Collector)
(Agent 2 Source -> FilesystemChannel -> Avro To Collector)
(Collector Avro from agents -> FilesystemChannel -> final Sink)

And perhaps in an even more complicated setup, with multiple collectors,
does that answer change further?

Viewing all articles
Browse latest Browse all 5648

Trending Articles