Anouncing Kafka Offset Monitor 0.1

Hello everyone,

at Quantifind, we are big users of Kafka and we like it a lot!
In a few use cases, we had to figure out if a queue was growing and how its
consumers were behaving. There are a few command-line tools to try to
figure out what's going on, but it's not always easy to debug and to see
what has happened while everyone was sleeping.

To be able to monitor our kafka queues and consumers, we thus developed a
tiny web app that could tell us the log size of each topic in the brokers
and the offsets of each consumers. That's very similar to what the kafka
ConsumerOffsetChecker tool* is doing, but instead of having a one off
snapshot, our app keeps an history and displays a nice graph of what's
going on.

You can find screenshots and more details here:
http://quantifind.github.io/KafkaOffsetMonitor/

the code is on github:
https://github.com/quantifind/KafkaOffsetMonitor

If you have kafka 0.8 setup, it's very easy to use:

1- download the current jar
http://quantifind.github.io/KafkaOffsetMonitor/dist/KafkaOffsetMonitor-assembly-0.1.0-SNAPSHOT.jar
2- run it, pointing at your kafka brokers:
java -cp KafkaOffsetMonitor-assembly-0.1.0-SNAPSHOT.jar \
com.quantifind.kafka.offsetapp.OffsetGetterWeb \
--zk zk-server1,zk-server2 \
--port 8080 \
--refresh 10.seconds \
--retain 2.days
3- open your browser and point it to localhost:8080

You can run it locally or host it on a server if you prefer.

It's all open source and we'll be happy to receive issue report and pull
requests.

I hope that you like it and that it can find some uses in the rest of the
Kafka community.

Best

Pierre

PS: we are aware of https://github.com/claudemamo/kafka-web-console but are
currently offering slightly different features. Hopefully we can merge the
projects in the future.

* here:
https://github.com/apache/kafka/blob/0.8/core/src/main/scala/kafka/tools/ConsumerOffsetChecker.scala