hi,
I am reading the book "Zookeeper" by Flavio Junqueira and Benjamin Reed.
And I am now concerned if Zookeeper right tool for our scenario:
configuration management. We have ~2000 servers that expected to subscribe
to znode change notification: current version number. As version number
changed all clients will read new value and read configuration
correspoinding this value:
/ currentVersion "v3"
/versions
/v1 {server1, server2, server3}
/v2 {server1, server2, server5}
/v3 {server0, server2, server3}
the idea we want to update configuration within seconds (<5s)
Is 2000 watch on same znode and than two simultaneous 2000 reads (one for
version and one for content) Ok for ZooKeeper?
according the book:
"...One issue to be aware of is that ZooKeeper triggers all watches set for
a particular znode change when the change occurs. If there are 1,000
clients that have set a watch on a given znode with a call to exists, then
1,000 notifications will be sent out when the znode is created. A change to
a watched znode might consequently generate a spike of notifica‐ tions.
Such a spike could affect, for example, the latency of operations submitted
around the time of the spike. When possible, we recommend avoiding such a
use of ZooKeeper in which a large number of clients watch for a change to a
given znode. It is much better to have only a few clients watching any
given znode at a time, and ideally at most one..."
1 vs 2000 is too big difference. And books says that even 1000 is a
problem. On other hand Zynga says that they did similar to our solution:
http://code.zynga.com/2011/08/updating-thousands-of-configuration-files-in-under-a-second/
Thank you,
Denis
I am reading the book "Zookeeper" by Flavio Junqueira and Benjamin Reed.
And I am now concerned if Zookeeper right tool for our scenario:
configuration management. We have ~2000 servers that expected to subscribe
to znode change notification: current version number. As version number
changed all clients will read new value and read configuration
correspoinding this value:
/ currentVersion "v3"
/versions
/v1 {server1, server2, server3}
/v2 {server1, server2, server5}
/v3 {server0, server2, server3}
the idea we want to update configuration within seconds (<5s)
Is 2000 watch on same znode and than two simultaneous 2000 reads (one for
version and one for content) Ok for ZooKeeper?
according the book:
"...One issue to be aware of is that ZooKeeper triggers all watches set for
a particular znode change when the change occurs. If there are 1,000
clients that have set a watch on a given znode with a call to exists, then
1,000 notifications will be sent out when the znode is created. A change to
a watched znode might consequently generate a spike of notifica‐ tions.
Such a spike could affect, for example, the latency of operations submitted
around the time of the spike. When possible, we recommend avoiding such a
use of ZooKeeper in which a large number of clients watch for a change to a
given znode. It is much better to have only a few clients watching any
given znode at a time, and ideally at most one..."
1 vs 2000 is too big difference. And books says that even 1000 is a
problem. On other hand Zynga says that they did similar to our solution:
http://code.zynga.com/2011/08/updating-thousands-of-configuration-files-in-under-a-second/
Thank you,
Denis