I'm seeing a NullPointerException in ZK 3.4.1, and it seems to usually
happen when I restart a server from standalone mode into quorum mode.  
It doesn't happen every time, but I can reproduce it pretty reliably in
my tests:

2011-12-19 14:33:04,036 2195 [pool-1-thread-2] INFO
org.apache.zookeeper.server.quorum.QuorumPeer  - tickTime set to 3000
2011-12-19 14:33:04,036 2195 [pool-1-thread-2] INFO
org.apache.zookeeper.server.quorum.QuorumPeer  - initLimit set to 5
2011-12-19 14:33:04,038 2197 [pool-1-thread-2] INFO
org.apache.zookeeper.server.persistence.FileSnap  - Reading snapshot
<zkdir>/version-2/snapshot.0
2011-12-19 14:33:04,040 2199 [pool-1-thread-2] DEBUG
org.apache.zookeeper.server.persistence.FileTxnLog  - Created new input
stream <zkdir>/version-2/log.1
2011-12-19 14:33:04,041 2200 [pool-1-thread-2] DEBUG
org.apache.zookeeper.server.persistence.FileTxnLog  - Created new input
archive <zkdir>/version-2/log.1
2011-12-19 14:33:04,041 2200 [pool-1-thread-2] DEBUG
org.apache.zookeeper.server.DataTree  - Ignoring processTxn failure hdr:
-1 : error: -101
java.lang.NullPointerException
         at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:203)
         at
org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
         at
org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
         at
org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
         at
org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
         at
com.nicira.onix.zookeeper.Zookeeper.StartZookeeper(Zookeeper.java:203)
         at
com.nicira.onix.zookeeper.Zookeeper.RestartZookeeper(Zookeeper.java:282)
         at com.nicira.onix.zookeeper.ZKRPCService.setServers(ZKRPC.java:83)
         at
com.nicira.onix.zookeeper.Zkrpc$ZKRPCService.callMethod(Zkrpc.java:8487)
         at com.nicira.onix.rpc.RPC$10.run(RPC.java:534)
         at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
         at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
         at java.lang.Thread.run(Thread.java:636)

(Note that I've replaced my zookeeper directory in the message above
with <zkdir> for readability.)

It appears to be a problem in the on-disk data, since if I then try
starting up a standalone server with the same directory, I get the same
error.  I've attached a tarball containing the version-2 directory that
demonstrates the problem.

If folks think this is a real problem in supported Zookeeper
configurations, I'll happily open a JIRA.

Jeremy

Sorry, I'd missed this message. I just created ZOOKEEPER-1333 for the
same issue.

-Andrew

On Mon, Dec 19, 2011 at 2:48 PM, Jeremy Stribling <[EMAIL PROTECTED]> wrote:
> I'm seeing a NullPointerException in ZK 3.4.1, and it seems to usually
> happen when I restart a server from standalone mode into quorum mode.  It
> doesn't happen every time, but I can reproduce it pretty reliably in my
> tests:
>
> 2011-12-19 14:33:04,036 2195 [pool-1-thread-2] INFO
> org.apache.zookeeper.server.quorum.QuorumPeer  - tickTime set to 3000
> 2011-12-19 14:33:04,036 2195 [pool-1-thread-2] INFO
> org.apache.zookeeper.server.quorum.QuorumPeer  - initLimit set to 5
> 2011-12-19 14:33:04,038 2197 [pool-1-thread-2] INFO
> org.apache.zookeeper.server.persistence.FileSnap  - Reading snapshot
> <zkdir>/version-2/snapshot.0
> 2011-12-19 14:33:04,040 2199 [pool-1-thread-2] DEBUG
> org.apache.zookeeper.server.persistence.FileTxnLog  - Created new input
> stream <zkdir>/version-2/log.1
> 2011-12-19 14:33:04,041 2200 [pool-1-thread-2] DEBUG
> org.apache.zookeeper.server.persistence.FileTxnLog  - Created new input
> archive <zkdir>/version-2/log.1
> 2011-12-19 14:33:04,041 2200 [pool-1-thread-2] DEBUG
> org.apache.zookeeper.server.DataTree  - Ignoring processTxn failure hdr: -1
> : error: -101
> java.lang.NullPointerException
>        at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:203)
>        at
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
>        at
> org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
>        at
> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
>        at
> org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
>        at
> com.nicira.onix.zookeeper.Zookeeper.StartZookeeper(Zookeeper.java:203)
>        at
> com.nicira.onix.zookeeper.Zookeeper.RestartZookeeper(Zookeeper.java:282)
>        at com.nicira.onix.zookeeper.ZKRPCService.setServers(ZKRPC.java:83)
>        at
> com.nicira.onix.zookeeper.Zkrpc$ZKRPCService.callMethod(Zkrpc.java:8487)
>        at com.nicira.onix.rpc.RPC$10.run(RPC.java:534)
>        at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>        at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>        at java.lang.Thread.run(Thread.java:636)
>
> (Note that I've replaced my zookeeper directory in the message above with
> <zkdir> for readability.)
>
> It appears to be a problem in the on-disk data, since if I then try starting
> up a standalone server with the same directory, I get the same error.  I've
> attached a tarball containing the version-2 directory that demonstrates the
> problem.
>
> If folks think this is a real problem in supported Zookeeper configurations,
> I'll happily open a JIRA.
>
> Jeremy
>

Thanks Andrew. This seems like a blocker.

Waiting for the patch :).

thanks
mahadev

On Mon, Dec 19, 2011 at 5:28 PM, Andrew McNair <[EMAIL PROTECTED]>wrote:

> Sorry, I'd missed this message. I just created ZOOKEEPER-1333 for the
> same issue.
>
> -Andrew
>
> On Mon, Dec 19, 2011 at 2:48 PM, Jeremy Stribling <[EMAIL PROTECTED]>
> wrote:
> > I'm seeing a NullPointerException in ZK 3.4.1, and it seems to usually
> > happen when I restart a server from standalone mode into quorum mode.  It
> > doesn't happen every time, but I can reproduce it pretty reliably in my
> > tests:
> >
> > 2011-12-19 14:33:04,036 2195 [pool-1-thread-2] INFO
> > org.apache.zookeeper.server.quorum.QuorumPeer  - tickTime set to 3000
> > 2011-12-19 14:33:04,036 2195 [pool-1-thread-2] INFO
> > org.apache.zookeeper.server.quorum.QuorumPeer  - initLimit set to 5
> > 2011-12-19 14:33:04,038 2197 [pool-1-thread-2] INFO
> > org.apache.zookeeper.server.persistence.FileSnap  - Reading snapshot
> > <zkdir>/version-2/snapshot.0
> > 2011-12-19 14:33:04,040 2199 [pool-1-thread-2] DEBUG
> > org.apache.zookeeper.server.persistence.FileTxnLog  - Created new input
> > stream <zkdir>/version-2/log.1
> > 2011-12-19 14:33:04,041 2200 [pool-1-thread-2] DEBUG
> > org.apache.zookeeper.server.persistence.FileTxnLog  - Created new input
> > archive <zkdir>/version-2/log.1
> > 2011-12-19 14:33:04,041 2200 [pool-1-thread-2] DEBUG
> > org.apache.zookeeper.server.DataTree  - Ignoring processTxn failure hdr:
> -1
> > : error: -101
> > java.lang.NullPointerException
> >        at
> >
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:203)
> >        at
> >
> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
> >        at
> > org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
> >        at
> >
> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
> >        at
> > org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
> >        at
> > com.nicira.onix.zookeeper.Zookeeper.StartZookeeper(Zookeeper.java:203)
> >        at
> > com.nicira.onix.zookeeper.Zookeeper.RestartZookeeper(Zookeeper.java:282)
> >        at
> com.nicira.onix.zookeeper.ZKRPCService.setServers(ZKRPC.java:83)
> >        at
> > com.nicira.onix.zookeeper.Zkrpc$ZKRPCService.callMethod(Zkrpc.java:8487)
> >        at com.nicira.onix.rpc.RPC$10.run(RPC.java:534)
> >        at
> >
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
> >        at
> >
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
> >        at java.lang.Thread.run(Thread.java:636)
> >
> > (Note that I've replaced my zookeeper directory in the message above with
> > <zkdir> for readability.)
> >
> > It appears to be a problem in the on-disk data, since if I then try
> starting
> > up a standalone server with the same directory, I get the same error.
>  I've
> > attached a tarball containing the version-2 directory that demonstrates
> the
> > problem.
> >
> > If folks think this is a real problem in supported Zookeeper
> configurations,
> > I'll happily open a JIRA.
> >
> > Jeremy
> >
>

Hi,

I've attached a failing test case to the issue. I didn't have time to
test the same thing in 3.3.x but I think it should pass.

Sorry, I initially added it as a comment then found i couldn't delete it.

-Andrew

On Mon, Dec 19, 2011 at 5:49 PM, Mahadev Konar <[EMAIL PROTECTED]> wrote:
> Thanks Andrew. This seems like a blocker.
>
> Waiting for the patch :).
>
> thanks
> mahadev
>
> On Mon, Dec 19, 2011 at 5:28 PM, Andrew McNair <[EMAIL PROTECTED]>wrote:
>
>> Sorry, I'd missed this message. I just created ZOOKEEPER-1333 for the
>> same issue.
>>
>> -Andrew
>>
>> On Mon, Dec 19, 2011 at 2:48 PM, Jeremy Stribling <[EMAIL PROTECTED]>
>> wrote:
>> > I'm seeing a NullPointerException in ZK 3.4.1, and it seems to usually
>> > happen when I restart a server from standalone mode into quorum mode.  It
>> > doesn't happen every time, but I can reproduce it pretty reliably in my
>> > tests:
>> >
>> > 2011-12-19 14:33:04,036 2195 [pool-1-thread-2] INFO
>> > org.apache.zookeeper.server.quorum.QuorumPeer  - tickTime set to 3000
>> > 2011-12-19 14:33:04,036 2195 [pool-1-thread-2] INFO
>> > org.apache.zookeeper.server.quorum.QuorumPeer  - initLimit set to 5
>> > 2011-12-19 14:33:04,038 2197 [pool-1-thread-2] INFO
>> > org.apache.zookeeper.server.persistence.FileSnap  - Reading snapshot
>> > <zkdir>/version-2/snapshot.0
>> > 2011-12-19 14:33:04,040 2199 [pool-1-thread-2] DEBUG
>> > org.apache.zookeeper.server.persistence.FileTxnLog  - Created new input
>> > stream <zkdir>/version-2/log.1
>> > 2011-12-19 14:33:04,041 2200 [pool-1-thread-2] DEBUG
>> > org.apache.zookeeper.server.persistence.FileTxnLog  - Created new input
>> > archive <zkdir>/version-2/log.1
>> > 2011-12-19 14:33:04,041 2200 [pool-1-thread-2] DEBUG
>> > org.apache.zookeeper.server.DataTree  - Ignoring processTxn failure hdr:
>> -1
>> > : error: -101
>> > java.lang.NullPointerException
>> >        at
>> >
>> org.apache.zookeeper.server.persistence.FileTxnSnapLog.processTransaction(FileTxnSnapLog.java:203)
>> >        at
>> >
>> org.apache.zookeeper.server.persistence.FileTxnSnapLog.restore(FileTxnSnapLog.java:150)
>> >        at
>> > org.apache.zookeeper.server.ZKDatabase.loadDataBase(ZKDatabase.java:223)
>> >        at
>> >
>> org.apache.zookeeper.server.quorum.QuorumPeer.loadDataBase(QuorumPeer.java:418)
>> >        at
>> > org.apache.zookeeper.server.quorum.QuorumPeer.start(QuorumPeer.java:410)
>> >        at
>> > com.nicira.onix.zookeeper.Zookeeper.StartZookeeper(Zookeeper.java:203)
>> >        at
>> > com.nicira.onix.zookeeper.Zookeeper.RestartZookeeper(Zookeeper.java:282)
>> >        at
>> com.nicira.onix.zookeeper.ZKRPCService.setServers(ZKRPC.java:83)
>> >        at
>> > com.nicira.onix.zookeeper.Zkrpc$ZKRPCService.callMethod(Zkrpc.java:8487)
>> >        at com.nicira.onix.rpc.RPC$10.run(RPC.java:534)
>> >        at
>> >
>> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1110)
>> >        at
>> >
>> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:603)
>> >        at java.lang.Thread.run(Thread.java:636)
>> >
>> > (Note that I've replaced my zookeeper directory in the message above with
>> > <zkdir> for readability.)
>> >
>> > It appears to be a problem in the on-disk data, since if I then try
>> starting
>> > up a standalone server with the same directory, I get the same error.
>>  I've
>> > attached a tarball containing the version-2 directory that demonstrates
>> the
>> > problem.
>> >
>> > If folks think this is a real problem in supported Zookeeper
>> configurations,
>> > I'll happily open a JIRA.
>> >
>> > Jeremy
>> >
>>