I have installed Accumulo 1.4.2 on top of Hadoop 0.20 and ZooKeeper 3.4.3 on CentOS 2.6.

I can start Hadoop namenode, datanodes, jobtracker, tasktrackers and 3-node zoo ensemble.  I can create things in Hadoop hdfs and zookeeper.  I did "accumulo init" without problems.

I can reach the accumulo web interface at http://x.x.x.x:50095/ which shows both tablet servers up and happy.  Log files everywhere are free of exceptions.

From "accumulo shell", I typed "createtable foobar".  One of the accumulo tabletserver log files shows:
2013-01-14 13:12:46,168 [server.TNonblockingServer] ERROR: Unexpected exception while invoking!
java.lang.NumberFormatException: For input string: "b53c3a3610ce0001"
at java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
at java.lang.Long.parseLong(Long.java:422)
at org.apache.accumulo.core.zookeeper.ZooUtil$LockID.<init>(ZooUtil.java:64)
at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.checkPermission(TabletServer.java:1794)
at org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.loadTablet(TabletServer.java:1814)
at org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor.process(TabletClientService.java:2037)
at org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:154)
at org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631)
at org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:202)

ZooUtil$LockID.<init>() on line 64 is:   "eid = Long.parseLong(sa[1], 16);"

so it's trying to convert b53c3a3610ce0001 to a long.

In a separate test file I did Long.parseLong("b53c3a3610ce0001", 16), which indeed throws NumberFormatException.

Long.MAX_LONG is 9223372036854775807, or 7FFFFFFFFFFFFFFF in hex.  b53c3a3610ce0001 is larger than this.

I can't see where this "serializedLID" parameter in ZooUtil comes from, or why it would be > Long.MAX_LONG.  Any ideas how to trace this and find out what is happening?

Thanks,
 - Rich

Wow... I've never seen this before.

The hex digit is the Zookeeper session id of the master.  The lock is
verified by the tablet server before it accepts the request to load a
tablet by the master.  It is encoded in LiveTServerSet.assignTablet.

I can't imagine how you would get an illegal long value, except if your
libraries didn't match across all your nodes.

-Eric

On Mon, Jan 14, 2013 at 3:34 PM, Chris Nafster <[EMAIL PROTECTED]>wrote:

> I have installed Accumulo 1.4.2 on top of Hadoop 0.20 and ZooKeeper 3.4.3
> on CentOS 2.6.
>
> I can start Hadoop namenode, datanodes, jobtracker, tasktrackers and
> 3-node zoo ensemble.  I can create things in Hadoop hdfs and zookeeper.  I
> did "accumulo init" without problems.
>
> I can reach the accumulo web interface at http://x.x.x.x:50095/ which
> shows both tablet servers up and happy.  Log files everywhere are free of
> exceptions.
>
> From "accumulo shell", I typed "createtable foobar".  One of the accumulo
> tabletserver log files shows:
> 2013-01-14 13:12:46,168 [server.TNonblockingServer] ERROR: Unexpected
> exception while invoking!
> java.lang.NumberFormatException: For input string: "b53c3a3610ce0001"
> at
> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
> at java.lang.Long.parseLong(Long.java:422)
> at
> org.apache.accumulo.core.zookeeper.ZooUtil$LockID.<init>(ZooUtil.java:64)
> at
> org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.checkPermission(TabletServer.java:1794)
> at
> org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.loadTablet(TabletServer.java:1814)
> at
> org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor.process(TabletClientService.java:2037)
> at
> org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:154)
> at
> org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631)
> at
> org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:202)
>
> ZooUtil$LockID.<init>() on line 64 is:   "eid = Long.parseLong(sa[1], 16);"
>
> so it's trying to convert b53c3a3610ce0001 to a long.
>
> In a separate test file I did Long.parseLong("b53c3a3610ce0001", 16),
> which indeed throws NumberFormatException.
>
> Long.MAX_LONG is 9223372036854775807, or 7FFFFFFFFFFFFFFF in hex.
>  b53c3a3610ce0001 is larger than this.
>
> I can't see where this "serializedLID" parameter in ZooUtil comes from, or
> why it would be > Long.MAX_LONG.  Any ideas how to trace this and find out
> what is happening?
>
> Thanks,
>  - Rich
>
>

On Mon, Jan 14, 2013 at 5:34 PM, Eric Newton <[EMAIL PROTECTED]> wrote:
> Wow... I've never seen this before.
>
> The hex digit is the Zookeeper session id of the master.  The lock is
> verified by the tablet server before it accepts the request to load a
> tablet by the master.  It is encoded in LiveTServerSet.assignTablet.
>
> I can't imagine how you would get an illegal long value, except if your
> libraries didn't match across all your nodes.
>
> -Eric
>

Seems like this may be a bug.  I looked at
LiveTServerSet.assignTablet() it eventually calls Long.toHexString().
The javadoc for toHexString() says the following.

     * Returns a string representation of the <code>long</code>
     * argument as an unsigned integer in base 16.

However, the method we are using to parse the Long can not handle
unsigned longs greater than MAX_LONG.  There does not seem to be a
method in long that can parse the output of  toHexString().  For
example the following will fail, any negative number will fail.

   Long.parseLong(Long.toHexString(-1), 16);

Rich, want to open a bug?  If not I can.

Keith

>
>
> On Mon, Jan 14, 2013 at 3:34 PM, Chris Nafster <[EMAIL PROTECTED]>wrote:
>
>> I have installed Accumulo 1.4.2 on top of Hadoop 0.20 and ZooKeeper 3.4.3
>> on CentOS 2.6.
>>
>> I can start Hadoop namenode, datanodes, jobtracker, tasktrackers and
>> 3-node zoo ensemble.  I can create things in Hadoop hdfs and zookeeper.  I
>> did "accumulo init" without problems.
>>
>> I can reach the accumulo web interface at http://x.x.x.x:50095/ which
>> shows both tablet servers up and happy.  Log files everywhere are free of
>> exceptions.
>>
>> From "accumulo shell", I typed "createtable foobar".  One of the accumulo
>> tabletserver log files shows:
>> 2013-01-14 13:12:46,168 [server.TNonblockingServer] ERROR: Unexpected
>> exception while invoking!
>> java.lang.NumberFormatException: For input string: "b53c3a3610ce0001"
>> at
>> java.lang.NumberFormatException.forInputString(NumberFormatException.java:48)
>> at java.lang.Long.parseLong(Long.java:422)
>> at
>> org.apache.accumulo.core.zookeeper.ZooUtil$LockID.<init>(ZooUtil.java:64)
>> at
>> org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.checkPermission(TabletServer.java:1794)
>> at
>> org.apache.accumulo.server.tabletserver.TabletServer$ThriftClientHandler.loadTablet(TabletServer.java:1814)
>> at
>> org.apache.accumulo.core.tabletserver.thrift.TabletClientService$Processor.process(TabletClientService.java:2037)
>> at
>> org.apache.accumulo.server.util.TServerUtils$TimedProcessor.process(TServerUtils.java:154)
>> at
>> org.apache.thrift.server.TNonblockingServer$FrameBuffer.invoke(TNonblockingServer.java:631)
>> at
>> org.apache.accumulo.server.util.TServerUtils$THsHaServer$Invocation.run(TServerUtils.java:202)
>>
>> ZooUtil$LockID.<init>() on line 64 is:   "eid = Long.parseLong(sa[1], 16);"
>>
>> so it's trying to convert b53c3a3610ce0001 to a long.
>>
>> In a separate test file I did Long.parseLong("b53c3a3610ce0001", 16),
>> which indeed throws NumberFormatException.
>>
>> Long.MAX_LONG is 9223372036854775807, or 7FFFFFFFFFFFFFFF in hex.
>>  b53c3a3610ce0001 is larger than this.
>>
>> I can't see where this "serializedLID" parameter in ZooUtil comes from, or
>> why it would be > Long.MAX_LONG.  Any ideas how to trace this and find out
>> what is happening?
>>
>> Thanks,
>>  - Rich
>>
>>

> Rich, want to open a bug?  If not I can.

Done!  https://issues.apache.org/jira/browse/ACCUMULO-965.  Please edit this to change whatever you feel is important.

To finish up this thread, is there a workaround for now?  I.e., can you control or affect the ZooKeeper session ID of the master, or just keep restarting until I get lucky?

 - Rich

I have never seen this.  What version of zookeeper are you running?

On Mon, Jan 14, 2013 at 6:28 PM, Chris Nafster <[EMAIL PROTECTED]> wrote:
>> Rich, want to open a bug?  If not I can.
>
> Done!  https://issues.apache.org/jira/browse/ACCUMULO-965.  Please edit this to change whatever you feel is important.
>
> To finish up this thread, is there a workaround for now?  I.e., can you control or affect the ZooKeeper session ID of the master, or just keep restarting until I get lucky?
>
>  - Rich
>