Hi,
I am trying to add new brokers to an existing 8 nodes Kafka cluster. We
have around 10 topics and the number of partition is set to 50. In order to
test the reassgin-partitions scripts, I tried on a sandbox cluster the
following steps.
I developed a script which is able to parse the reassignment partition plan
given by the Kafka tool in smaller pieces (reassigning maximum 10
partitions at a time).
Unfortunately I faced some issues with the tool that sometimes get stuck on
one partition. In this case I have to kill and restart the three Kafkas on
which the partition has been relocated to unlock the process (One kafka at
a time).
Moreover, I have also faced these two issues that are already on Jira:
https://issues.apache.org/jira/browse/KAFKA-1382
https://issues.apache.org/jira/browse/KAFKA-1479
We really need to add new nodes to our Kafka cluster, does anybody have
already rebalance a Kafka 0.8.1.1? What could you advise me?
Thanks, and feel free to ask me if you need more details.
I am trying to add new brokers to an existing 8 nodes Kafka cluster. We
have around 10 topics and the number of partition is set to 50. In order to
test the reassgin-partitions scripts, I tried on a sandbox cluster the
following steps.
I developed a script which is able to parse the reassignment partition plan
given by the Kafka tool in smaller pieces (reassigning maximum 10
partitions at a time).
Unfortunately I faced some issues with the tool that sometimes get stuck on
one partition. In this case I have to kill and restart the three Kafkas on
which the partition has been relocated to unlock the process (One kafka at
a time).
Moreover, I have also faced these two issues that are already on Jira:
https://issues.apache.org/jira/browse/KAFKA-1382
https://issues.apache.org/jira/browse/KAFKA-1479
We really need to add new nodes to our Kafka cluster, does anybody have
already rebalance a Kafka 0.8.1.1? What could you advise me?
Thanks, and feel free to ask me if you need more details.