Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Standalone session expiration


+
Jared Cantwell 2011-06-30, 15:15
+
Ted Dunning 2011-07-02, 16:55
Copy link to this message
-
Re: Standalone session expiration
Forgot to reply to this. It looks like the socket linger issue was  
causing this. I didn't track it completely since disabling linger  
solved the issue. My guess is that the sockets were hung on close for  
2 seconds each, which somehow caused the server to kill other  
connections (maybe a lock is held?).

Jared

On Jul 2, 2011, at 10:55 AM, Ted Dunning <[EMAIL PROTECTED]> wrote:

> Has anybody understood this scenario yet?
>
> On Thu, Jun 30, 2011 at 8:15 AM, Jared Cantwell <[EMAIL PROTECTED]
> >wrote:
>
>> Hello again,
>>
>> I am seeing a strange issue that I'm hoping someone can give me  
>> insight
>> into.  For simple testing, I have a standalone server setup.  I  
>> connected
>> to
>> this server from 2 nodes, one of which is the node hosting the  
>> standalone
>> server.  After opening a small number of connections from each node  
>> (3 or 4
>> clients/node), I powered off the node not hosting the standalone  
>> server.
>> As
>> expected, the logs show the server expiring all sessions for  
>> connections to
>> that node.  The problem comes 10 seconds later when the server  
>> decides to
>> also expire all local connections too.  As a result, the clients on  
>> the
>> node
>> that is still alive (and hosting the standalone server) all try
>> reconnecting, but their connections are denied for having expired--  
>> over
>> and
>> over again.
>>
>> I am working on getting some consolidated logs, so I'll reply to  
>> this when
>> I
>> have them.  I was wondering if anyone knows of an issue or has any  
>> initial
>> thoughts?
>>
>> Some things I am going to try:
>> 1. Start a 3 node quorum and connect clients from a 4th node.  Then  
>> kill
>> the
>> 4th node and see if other connections are killed too.  If this  
>> works OK
>> then
>> it would point to an issue with the standalone server mode.
>> 2. Connect 3 nodes to my standalone server.  Power off one node and  
>> see if
>> connections to the other node is killed.  This will determine if its
>> killing
>> all other connections, or just local connections for some strange  
>> reasons.
>>
>> ~Jared
>>
+
Ted Dunning 2011-07-02, 17:02
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB