Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # user - Question regarding the maximum number of ZNODES a zookeeper


Copy link to this message
-
Re: Question regarding the maximum number of ZNODES a zookeeper
Vitalii Tymchyshyn 2011-11-10, 10:48
10.11.11 06:53, Aniket Chakrabarti написав(ла):
> Hi Patrick,
>
> I tried setting the max heap size to 2G, but still after creating 1027
> x 480 nodes zookeeper is throwing the error code "-5". Following is
> the exception I'm seeing:
>
> -----------------------------------------------------------------------------------------------------------------------------------------------
>
> 2011-11-09 23:42:49,484 - ERROR
> [ProcessThread:-1:PrepRequestProcessor@415] - Failed to process
> sessionid:0x1338bb71d340000 type:create cxid:0x4ec3599e
> zxid:0xfffffffffffffffe txntype:unknown reqpath:/4/158:1096
> java.nio.BufferOverflowException
> at java.nio.charset.CoderResult.throwException(Unknown Source)
> at java.lang.StringCoding$StringDecoder.decode(Unknown Source)
> at java.lang.StringCoding.decode(Unknown Source)
> at java.lang.String.<init>(Unknown Source)
> at java.lang.String.<init>(Unknown Source)
> at
> org.apache.jute.BinaryInputArchive.readString(BinaryInputArchive.java:83)
> at org.apache.zookeeper.data.Id.deserialize(Id.java:55)
> at
> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108)
> at org.apache.zookeeper.data.ACL.deserialize(ACL.java:57)
> at
> org.apache.jute.BinaryInputArchive.readRecord(BinaryInputArchive.java:108)
> at
> org.apache.zookeeper.proto.CreateRequest.deserialize(CreateRequest.java:92)
> at
> org.apache.zookeeper.server.ZooKeeperServer.byteBuffer2Record(ZooKeeperServer.java:599)
> at
> org.apache.zookeeper.server.PrepRequestProcessor.pRequest(PrepRequestProcessor.java:216)
> at
> org.apache.zookeeper.server.PrepRequestProcessor.run(PrepRequestProcessor.java:114)
> 2011-11-09 23:42:49,485 - ERROR
> [ProcessThread:-1:PrepRequestProcessor@428] - Dumping request buffer:
> 0x000b2f342f3135383a3130393600013100010001f0005776f726c640006616e796f6e650000
> -----------------------------------------------------------------------------------------------------------------------------
>
>
> Please advise.

There is a limit on how many nodes can single parent node have if you
use list operation. The limit is not direct, but rather it's a limit on
response size. It can be changed (don't remember if with a setting or
compile time).

Best regards, Vitalii Tymchyshyn