Using Kafka 0.8, when specifying a starting offset and appropriate
fetchSize, SimpleConsumer will only return up to the highest offset of the
log segment containing the starting offset.
For example,
log segment #1 contains offsets 1 - 10
log segment #2 contiains offsets 11 - 100
A fetch request specifying offset 9, will only return offsets 9 and 10.
Specifying starting offset 11 will return 11 - 100
Must one check fetchResponse.highwatermark each time to determine if there
are higher numbered offsets, and continue submitting fetchRequests ?
I assumed SimpleConsumer would return offsets/messages spanning log
segments, up to the specified fetchSize.
Thanks!
fetchSize, SimpleConsumer will only return up to the highest offset of the
log segment containing the starting offset.
For example,
log segment #1 contains offsets 1 - 10
log segment #2 contiains offsets 11 - 100
A fetch request specifying offset 9, will only return offsets 9 and 10.
Specifying starting offset 11 will return 11 - 100
Must one check fetchResponse.highwatermark each time to determine if there
are higher numbered offsets, and continue submitting fetchRequests ?
I assumed SimpleConsumer would return offsets/messages spanning log
segments, up to the specified fetchSize.
Thanks!