Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

consumer partition rebalancing

$
0
0
I wasreviewingtheconsumer partition rebalancingalgorithm and had a fewrelatedquestions 

* Assuming 1) theconsumerdoesn'tcontrolthe partition allocation within a topic and 2) theconstraintthat a single consumer C(i) within a consumer group C(g) must be the only reader of that partition:
* Are there ways to scale partition consumption if C(i) cannot keep up?
* A way to designate particular consumers should consume particular partitions based on consumer capability (similar to container allocation in YARN)?
* A way to designate sticky consumer / partition allocation so that a new consumer C(j) or new broker B(k) or new topic T(l) does not cause C(i) to lose particular partitions during rebalancing?

Thanks
Kam

Viewing all articles
Browse latest Browse all 5648

Trending Articles