Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Question regarding the maximum number of ZNODES a zookeeper


+
Aniket Chakrabarti 2011-11-09, 21:14
+
Patrick Hunt 2011-11-09, 22:31
+
Patrick Hunt 2011-11-09, 23:00
+
Aniket Chakrabarti 2011-11-10, 04:53
Copy link to this message
-
Re: Question regarding the maximum number of ZNODES a zookeeper
10.11.11 06:53, Aniket Chakrabarti написав(ла):
> Hi Patrick,
>
> I tried setting the max heap size to 2G, but still after creating 1027
> x 480 nodes zookeeper is throwing the error code "-5". Following is
> the exception I'm seeing:
>
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> 2011-11-09 23:42:49,484 - ERROR
> [ProcessThread:-1:PrepRequestProcessor@415] - Failed to process
> sessionid:0x1338bb71d340000 type:create cxid:0x4ec3599e
> zxid:0xfffffffffffffffe txntype:unknown reqpath:/4/158:1096
> java.nio.BufferOverflowException
> at java.nio.charset.CoderResult.throwException(Unknown Source)
> at java.lang.StringCoding$StringDecoder.decode(Unknown Source)
> at java.lang.StringCoding.decode(Unknown Source)
> at java.lang.String.<init>(Unknown Source)
> at java.lang.String.<init>(Unknown Source)
> at
> org.apache.jute.BinaryInputArchive.readString(BinaryInputArchive.java:83)
> at org.apache.zookeeper.data.Id.deserialize(Id.java:55)
> at
> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108)
> at org.apache.zookeeper.data.ACL.deserialize(ACL.java:57)
> at
> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108)
> at
> org.apache.zookeeper.proto.CreateRequest.deserialize(CreateRequest.java:92)
> at
> org.apache.zookeeper.server.ZooKeeperServer.byteBuffer2Record(ZooKeeperServer.java:599)
> at
> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:216)
> at
> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114)
> 2011-11-09 23:42:49,485 - ERROR
> [ProcessThread:-1:PrepRequestProcessor@428] - Dumping request buffer:
> 0x000b2f342f3135383a3130393600013100010001f0005776f726c640006616e796f6e650000
> -----------------------------------------------------------------------------------------------------------------------------
>
>
> Please advise.

There is a limit on how many nodes can single parent node have if you
use list operation. The limit is not direct, but rather it's a limit on
response size. It can be changed (don't remember if with a setting or
compile time).

Best regards, Vitalii Tymchyshyn
+
Patrick Hunt 2011-11-10, 17:34
+
Ted Dunning 2011-11-10, 05:07
+
Patrick Hunt 2011-11-10, 06:18
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB