Quantcast
Channel: Apache Timeline
Viewing all articles
Browse latest Browse all 5648

Facing issues with Kafka 0.8.1.1 and kafka-reassign-partitions.sh

$
0
0
Hi,

I am trying to add new brokers to an existing 8 nodes Kafka cluster. We
have around 10 topics and the number of partition is set to 50. In order to
test the reassgin-partitions scripts, I tried on a sandbox cluster the
following steps.

I developed a script which is able to parse the reassignment partition plan
given by the Kafka tool in smaller pieces (reassigning maximum 10
partitions at a time).

Unfortunately I faced some issues with the tool that sometimes get stuck on
one partition. In this case I have to kill and restart the three Kafkas on
which the partition has been relocated to unlock the process (One kafka at
a time).

Moreover, I have also faced these two issues that are already on Jira:

https://issues.apache.org/jira/browse/KAFKA-1382
https://issues.apache.org/jira/browse/KAFKA-1479

We really need to add new nodes to our Kafka cluster, does anybody have
already rebalance a Kafka 0.8.1.1? What could you advise me?

Thanks, and feel free to ask me if you need more details.

Viewing all articles
Browse latest Browse all 5648

Latest Images

Trending Articles



Latest Images