Hi,
My team trying to implement lambda architecture.
We need to stream all our new data though Kafka to storm, and HDFS.
As i see it were are two options:
Using Camus - not very efficent
Streaming via Storm - not very efficent
Is it possible to persist the queue's files over the HDFS (with short-circuit setting switched on) instead of the local filesystem?
It suppose to have similar performance to the local file system, doesn't it?
My team trying to implement lambda architecture.
We need to stream all our new data though Kafka to storm, and HDFS.
As i see it were are two options:
Using Camus - not very efficent
Streaming via Storm - not very efficent
Is it possible to persist the queue's files over the HDFS (with short-circuit setting switched on) instead of the local filesystem?
It suppose to have similar performance to the local file system, doesn't it?