Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Facing issues with Kafka 0.8.1.1 and kafka-reassign-partitions.sh

$
0
0
Hi,

I am trying to add new brokers to an existing 8 nodes Kafka cluster. We
have around 10 topics and the number of partition is set to 50. In order to
test the reassgin-partitions scripts, I tried on a sandbox cluster the
following steps.

I developed a script which is able to parse the reassignment partition plan
given by the Kafka tool in smaller pieces (reassigning maximum 10
partitions at a time).

Unfortunately I faced some issues with the tool that sometimes get stuck on
one partition. In this case I have to kill and restart the three Kafkas on
which the partition has been relocated to unlock the process (One kafka at
a time).

Moreover, I have also faced these two issues that are already on Jira:

https://issues.apache.org/jira/browse/KAFKA-1382
https://issues.apache.org/jira/browse/KAFKA-1479

We really need to add new nodes to our Kafka cluster, does anybody have
already rebalance a Kafka 0.8.1.1? What could you advise me?

Thanks, and feel free to ask me if you need more details.

Viewing all articles
Browse latest Browse all 5648

Trending Articles