Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

How to connect two flows with specific behavior.

$
0
0
I want to create a topology for Flume, what I want to get it's,.

Data---> Source1-->Channel1-->MySink1 --->Source2 --> Channel2/Channel3

Channel2 --> SinkHDFS
Channel3 --> MySinkHBase

I'd need to code MySink1 and do an special transformation to my data, the
output would be the input for Source2.
Finally, these data should store in Hdfs with the standard sink of flume
and HBase, where I should create a new Serializer for HBase or something
like that.

I can't see how to do the connection between MySink1 and Source2.
Should Source2 be of Avro type? I think that if I want to connect many
flows inside Flume they have to be Avro, How I want an specific behavior, I
should create a new implementation which extends AbstractRpcAvro or
something like that... Am I right?

Viewing all articles
Browse latest Browse all 5648

Trending Articles