Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Newbie - Sink question

$
0
0
Hello All,
We have the following configuration:
Source->Channel->Sink

Now, the source is pointing to a folder that has lots of json files. The
channel is file based so that there is fault tolerance and the Sink is
putting CSV files on S3.

Now, there is code written in Sink that takes the JSON events and does some
MySQL database lookup and generates CSV files to be put into S3.

The question is, is it the right place for the code or should the code be
running in channel as the ACID gaurantees is present in Channel. Please
advise.

-Kev

Viewing all articles
Browse latest Browse all 5648

Trending Articles