Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - very uneven distribution of clients to servers...


Copy link to this message
-
Re: very uneven distribution of clients to servers...
Brian Tarbox 2012-12-06, 11:32
Thanks for all the thoughts.

We are using the java client.
I killed all the clients and then restarted them do its not a reconnection issue .

Sent from my iPhone

On Dec 5, 2012, at 6:15 PM, Alexander Shraer <[EMAIL PROTECTED]> wrote:

> I have some experience with this that may be useful (Marshall and I
> worked on the re-shuffling of clients to servers as part of ZK-1355).
>
> Kishore's suggestion is the first thing I would check, but there is
> another possibility.
>
> We were seeing the same issue - the distribution of clients across
> servers was not even (we allow some slack in the tests, but still it
> would frequently be very uneven). The reason ended up being that
> different clients were shuffling the list in the same way, and when I
> incorporated the client's id into the seed the problem went away. I'm
> curious whether this is the problem in this case too - this can be
> easily tested by trying whether the problem is still there with the
> trunk distribution (ZK-1355 is in the trunk) or just applying ZK-1355.
>
> We're currently seeing the same thing with our C tests:
> https://issues.apache.org/jira/browse/ZOOKEEPER-1594
> But haven't yet tried whether the same solution would work there.
>
> Alex
>
>
> On Wed, Dec 5, 2012 at 12:54 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:
>> Shuffle depends on Math.random which is seeded by time of start.  That
>> should be just fine.
>>
>> On Wed, Dec 5, 2012 at 11:35 PM, Camille Fournier <[EMAIL PROTECTED]>wrote:
>>
>>> If you're using the Java ZooKeeper client, you can see in the code that the
>>> way connections are established is that we parse the server list, resolve
>>> them to inet addresses, and call Collections.shuffle on the list of server
>>> addresses. It's possible that Collections.shuffle is not random enough but
>>> I suspect that there's something else happening.
>>>