Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> very uneven distribution of clients to servers...


+
Brian Tarbox 2012-12-05, 20:27
+
kishore g 2012-12-05, 22:01
+
Ted Dunning 2012-12-05, 22:53
+
Camille Fournier 2012-12-05, 22:35
+
Ted Dunning 2012-12-05, 22:54
Copy link to this message
-
Re: very uneven distribution of clients to servers...
I have some experience with this that may be useful (Marshall and I
worked on the re-shuffling of clients to servers as part of ZK-1355).

Kishore's suggestion is the first thing I would check, but there is
another possibility.

We were seeing the same issue - the distribution of clients across
servers was not even (we allow some slack in the tests, but still it
would frequently be very uneven). The reason ended up being that
different clients were shuffling the list in the same way, and when I
incorporated the client's id into the seed the problem went away. I'm
curious whether this is the problem in this case too - this can be
easily tested by trying whether the problem is still there with the
trunk distribution (ZK-1355 is in the trunk) or just applying ZK-1355.

We're currently seeing the same thing with our C tests:
https://issues.apache.org/jira/browse/ZOOKEEPER-1594
But haven't yet tried whether the same solution would work there.

Alex
On Wed, Dec 5, 2012 at 12:54 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:
> Shuffle depends on Math.random which is seeded by time of start.  That
> should be just fine.
>
> On Wed, Dec 5, 2012 at 11:35 PM, Camille Fournier <[EMAIL PROTECTED]>wrote:
>
>> If you're using the Java ZooKeeper client, you can see in the code that the
>> way connections are established is that we parse the server list, resolve
>> them to inet addresses, and call Collections.shuffle on the list of server
>> addresses. It's possible that Collections.shuffle is not random enough but
>> I suspect that there's something else happening.
>>
+
Brian Tarbox 2012-12-06, 11:32
+
Ted Dunning 2012-12-07, 01:25
+
Alexander Shraer 2012-12-07, 01:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB