Hello All,
We have the following configuration:
Source->Channel->Sink
Now, the source is pointing to a folder that has lots of json files. The
channel is file based so that there is fault tolerance and the Sink is
putting CSV files on S3.
Now, there is code written in Sink that takes the JSON events and does some
MySQL database lookup and generates CSV files to be put into S3.
The question is, is it the right place for the code or should the code be
running in channel as the ACID gaurantees is present in Channel. Please
advise.
-Kev
We have the following configuration:
Source->Channel->Sink
Now, the source is pointing to a folder that has lots of json files. The
channel is file based so that there is fault tolerance and the Sink is
putting CSV files on S3.
Now, there is code written in Sink that takes the JSON events and does some
MySQL database lookup and generates CSV files to be put into S3.
The question is, is it the right place for the code or should the code be
running in channel as the ACID gaurantees is present in Channel. Please
advise.
-Kev