Hi Group,
We've implemented custom Flume Sink.
We do some DB lookups for sanity checks within it and bunch of stuff in
order to slightly prepare data for insertion into the postgresql via copy
command.
It gets the channel, transaction and it executes begin() method.
Of course, there is try-catch-finally block wrapper that ensures commit or
rollback methods are called and finally transaction is closed.
Sometimes it happens that there is an exception due to transaction.begin
called before transaction.commit or rollback.
I don't see it possible unless there are two threads take the same
transaction.
Another possible reason might be short code in the catch block that could
prevent call to the transaction.rollback and transaction.close.
However, it doesn't throw an exception within catch block as it would
emerge in logs.
This is relevant part of the code:
@Override
public Status process() throws EventDeliveryException {
Channel channel = getChannel();
Transaction transaction = channel.getTransaction();
...
try {
transaction.begin();
event = channel.take();
...
transaction.commit();
} catch (Exception ex) {
LOG.error("Problem with sink", ex);
LOG.error("Exception: " + ex.toString());
LOG.info("Start source...");
this.cacheDBPool.clear();
CommandExecutor.execUnStopCmd(this.flumeFolderLogs);
transaction.rollback();
} finally {
transaction.close();
return result;
}// end method
Thanks in advance
Best regards,
Ahmed Vila
We've implemented custom Flume Sink.
We do some DB lookups for sanity checks within it and bunch of stuff in
order to slightly prepare data for insertion into the postgresql via copy
command.
It gets the channel, transaction and it executes begin() method.
Of course, there is try-catch-finally block wrapper that ensures commit or
rollback methods are called and finally transaction is closed.
Sometimes it happens that there is an exception due to transaction.begin
called before transaction.commit or rollback.
I don't see it possible unless there are two threads take the same
transaction.
Another possible reason might be short code in the catch block that could
prevent call to the transaction.rollback and transaction.close.
However, it doesn't throw an exception within catch block as it would
emerge in logs.
This is relevant part of the code:
@Override
public Status process() throws EventDeliveryException {
Channel channel = getChannel();
Transaction transaction = channel.getTransaction();
...
try {
transaction.begin();
event = channel.take();
...
transaction.commit();
} catch (Exception ex) {
LOG.error("Problem with sink", ex);
LOG.error("Exception: " + ex.toString());
LOG.info("Start source...");
this.cacheDBPool.clear();
CommandExecutor.execUnStopCmd(this.flumeFolderLogs);
transaction.rollback();
} finally {
transaction.close();
return result;
}// end method
Thanks in advance
Best regards,
Ahmed Vila