Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> zooKeeper connectionloss on a folder


+
Yuhan Zhang 2012-05-01, 03:24
+
Ted Dunning 2012-05-01, 05:50
+
Yuhan Zhang 2012-05-01, 17:17
+
Ted Dunning 2012-05-01, 18:22
Copy link to this message
-
Re: zooKeeper connectionloss on a folder
Hi Ted,

Thanks for the quick response. could you direct me to the part of the
zookeeper doc
for configuring this limit?

The Curator DistributedQueue watches for changes in the childNodes of a
node.
I'm depending on this to place items onto the queue, and make other
machines to
be notified and start processing. It would be a lot of work to modify the
queue to
watch on multiple dymanically-created nodes.. will think about this
approach later.
Thank you.

Yuhan

On Tue, May 1, 2012 at 11:22 AM, Ted Dunning <[EMAIL PROTECTED]> wrote:

> There is a size limit for responses from the server.  That is likely the
> problem you are seeing.  This can be configured larger, but that isn't
> really a solution.
>
> It would be better to address this at the architectural level.  Is there
> some way that you can go to a multi-level directory structure?
>
> On Tue, May 1, 2012 at 10:17 AM, Yuhan Zhang <[EMAIL PROTECTED]> wrote:
>
> > More than 100000 objects under a node... I restarted last night with an
> > empty folder,
> > and the new folder reaches the same state. It kept happening..
> >
> > I'm using the curator library to add objects.
> > (com.netflix.curator.framework.recipes.queue.DistributedQueue)  But
> feeling
> > it is a
> > problem of ZooKeeper.
> >
> > Thank you.
> >
> > Yuhan
> >
> > On Mon, Apr 30, 2012 at 10:50 PM, Ted Dunning <[EMAIL PROTECTED]>
> > wrote:
> >
> > > How many objects are in that directory?
> > >
> > > Sent from my iPhone
> > >
> > > On Apr 30, 2012, at 8:24 PM, Yuhan Zhang <[EMAIL PROTECTED]> wrote:
> > >
> > > > Hi all,
> > > >
> > > > I'm using zooKeeper as a task queue; however, it appears that the
> > folder
> > > > (zookeeper data node) that I used to store the tasks
> > > > (zooKeeper data nodes) are not accessible anymore - whenever I run an
> > > "ls"
> > > > through zkCli.sh, the app will quit with a connectionLoss.
> > > >
> > > > The other folders are ok. and viewable using "ls".
> > > >
> > > > Is the file corrupted? There are a lot of objects in the files. Is
> > there
> > > a
> > > > limit for the number of object can be stored in a folder (node)?
> > > >
> > > >
> > > > Thank you.
> > > >
> > > > Yuhan Zhang
> > > >
> > > > Here are the error messages:
> > > > [zk: localhost:2181(CONNECTED) 1] ls /com.onescreen.viewer
> > > > 2012-05-01 03:21:39,533 - WARN
> > > > [main-SendThread(localhost:2181):ClientCnxn$SendThread@1161] -
> Session
> > > > 0x1370667c2210003 for server localhost/127.0.0.1:2181, unexpected
> > error,
> > > > closing socket connection and attempting reconnect
> > > > java.io.IOException: Packet len4315700 is out of range!
> > > >    at
> > > >
> > >
> >
> org.apache.zookeeper.ClientCnxn$SendThread.readLength(ClientCnxn.java:710)
> > > >    at
> > > org.apache.zookeeper.ClientCnxn$SendThread.doIO(ClientCnxn.java:869)
> > > >    at
> > > org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1130)
> > > >
> > > > WATCHER::
> > > >
> > > > WatchedEvent state:Disconnected type:None path:null
> > > > Exception in thread "main"
> > > > org.apache.zookeeper.KeeperException$ConnectionLossException:
> > > > KeeperErrorCode = ConnectionLoss for /com.onescreen.viewer
> > > >    at
> > > org.apache.zookeeper.KeeperException.create(KeeperException.java:90)
> > > >    at
> > > org.apache.zookeeper.KeeperException.create(KeeperException.java:42)
> > > >    at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1243)
> > > >    at org.apache.zookeeper.ZooKeeper.getChildren(ZooKeeper.java:1271)
> > > >    at
> > > >
> org.apache.zookeeper.ZooKeeperMain.processZKCmd(ZooKeeperMain.java:700)
> > > >    at
> > > org.apache.zookeeper.ZooKeeperMain.processCmd(ZooKeeperMain.java:581)
> > > >    at
> > > >
> org.apache.zookeeper.ZooKeeperMain.executeLine(ZooKeeperMain.java:353)
> > > >    at org.apache.zookeeper.ZooKeeperMain.run(ZooKeeperMain.java:311)
> > > >    at org.apache.zookeeper.ZooKeeperMain.main(ZooKeeperMain.java:270)
> > >
> >
> >
> >
> > --
> > Yuhan Zhang
> > Application Developer

Yuhan Zhang
Application Developer
OneScreen Inc.
[EMAIL PROTECTED] <[EMAIL PROTECTED]>
www.onescreen.com

The information contained in this e-mail is for the exclusive use of the
intended recipient(s) and may be confidential, proprietary, and/or legally
privileged. Inadvertent disclosure of this message does not constitute a
waiver of any privilege.  If you receive this message in error, please do
not directly or indirectly print, copy, retransmit, disseminate, or
otherwise use the information. In addition, please delete this e-mail and
all copies and notify the sender.