We do use the file component to read files from a particular area. We pick up
new files from there using their content as basis for a database import.
There are a couple of important points:
1. We must not add or delete files to the area we read from, not even
temporarily.
2. We need to pick up files once.
To achive this we use noop=true as well as a file based idempotentRepository
where the repository is placed in a different location. We also use
readLock=changed which is the big problem. To cite the component
documentation:
Notice from Camel 2.10 onwards the read locks changed, fileLock and rename
will also use a markerFile as well, to ensure not picking up files that may
be in process by another Camel consumer running on another node (eg
cluster).
This is a huge problem as I can't find any documented way of storing the
marker files in a different location. What is really needed is something
similar to inProgressRepository as an option to keep track of the readLock
related information.
This problem is for us serious breakage, as there are other applications not
under our control reading the same areas, some of these crashes due to the
marker files.
new files from there using their content as basis for a database import.
There are a couple of important points:
1. We must not add or delete files to the area we read from, not even
temporarily.
2. We need to pick up files once.
To achive this we use noop=true as well as a file based idempotentRepository
where the repository is placed in a different location. We also use
readLock=changed which is the big problem. To cite the component
documentation:
Notice from Camel 2.10 onwards the read locks changed, fileLock and rename
will also use a markerFile as well, to ensure not picking up files that may
be in process by another Camel consumer running on another node (eg
cluster).
This is a huge problem as I can't find any documented way of storing the
marker files in a different location. What is really needed is something
similar to inProgressRepository as an option to keep track of the readLock
related information.
This problem is for us serious breakage, as there are other applications not
under our control reading the same areas, some of these crashes due to the
marker files.