Hardware planning

Hi all,

I'm looking at setting up a (small) Kafka cluster for streaming microscope data to Spark-Streaming.

The producer would be a single Windows 7 machine with a 1Gb or 10Gb ethernet connection running http posts from Matlab (this bit is a little fuzzy, and I'm not the user, I'm an admin), the consumer would be 10-60 (or more) Linux nodes running Spark-Streaming with 10Gb ethernet connections. Target data rate per the user is <200MB/sec, although I can see this scaling in the future.

Based on the documentation, my initial thoughts were as follows:

3 nodes, all running ZK and the broker

Dell R620
2x8 core 2.6GHz Xeon
256GB RAM
8x300GB 15K SAS drives (OS runs on 2, ZK on 1, broker on the last 5)
10Gb ethernet (single port)

Do these specs make sense? Am I over or under-speccing in any of the areas? It made sense to me to make the filesystem cache as large as possible, particularly when I'm dealing with a small number of brokers.

Thanks,
Ken Carlile
Senior Unix Engineer, Scientific Computing Systems
Janelia Farm Research Campus, HHMI