Has anyone had an issue attempting to delete an empty node (zk 3.4.3):

[zk: localhost:2181(CONNECTED) 252] ls /streams/foo
[]
[zk: localhost:2181(CONNECTED) 253] delete /streams/foo
Node not empty: /streams/foo
[zk: localhost:2181(CONNECTED) 254] get /streams/foo

cZxid = 0x245f
ctime = Thu Mar 08 18:36:12 PST 2012
mZxid = 0x245f
mtime = Thu Mar 08 18:36:12 PST 2012
pZxid = 0x27c0
cversion = 38
dataVersion = 0
aclVersion = 0
ephemeralOwner = 0x0
dataLength = 0
numChildren = 0

This doesn't seem to happen consistently, but more often when under load.

Timing? Perhaps by the time you get around to doing the get the child
has been removed, but one/more exist at the time you do the delete.
Notice the cversion is 38 in this case (so there were children at one
point).

Patrick

On Fri, Mar 9, 2012 at 11:40 AM, Shelley, Ryan <[EMAIL PROTECTED]> wrote:
> Has anyone had an issue attempting to delete an empty node (zk 3.4.3):
>
> [zk: localhost:2181(CONNECTED) 252] ls /streams/foo
> []
> [zk: localhost:2181(CONNECTED) 253] delete /streams/foo
> Node not empty: /streams/foo
> [zk: localhost:2181(CONNECTED) 254] get /streams/foo
>
> cZxid = 0x245f
> ctime = Thu Mar 08 18:36:12 PST 2012
> mZxid = 0x245f
> mtime = Thu Mar 08 18:36:12 PST 2012
> pZxid = 0x27c0
> cversion = 38
> dataVersion = 0
> aclVersion = 0
> ephemeralOwner = 0x0
> dataLength = 0
> numChildren = 0
>
> This doesn't seem to happen consistently, but more often when under load.

There were children previously, yes, but I tried to do the delete well
after the ephemeral children had been removed. I stopped my services (so
ephemeral children were removed), did an "ls" which showed the children
had been removed, try the "delete" which reported an error, then I did a
"get" just to show the node and that it's reporting 0 children. No
services were running against ZK while I tried the delete as I had stopped
them all.

On 3/9/12 11:43 AM, "Patrick Hunt" <[EMAIL PROTECTED]> wrote:

>Timing? Perhaps by the time you get around to doing the get the child
>has been removed, but one/more exist at the time you do the delete.
>Notice the cversion is 38 in this case (so there were children at one
>point).
>
>Patrick
>
>On Fri, Mar 9, 2012 at 11:40 AM, Shelley, Ryan <[EMAIL PROTECTED]>
>wrote:
>> Has anyone had an issue attempting to delete an empty node (zk 3.4.3):
>>
>> [zk: localhost:2181(CONNECTED) 252] ls /streams/foo
>> []
>> [zk: localhost:2181(CONNECTED) 253] delete /streams/foo
>> Node not empty: /streams/foo
>> [zk: localhost:2181(CONNECTED) 254] get /streams/foo
>>
>> cZxid = 0x245f
>> ctime = Thu Mar 08 18:36:12 PST 2012
>> mZxid = 0x245f
>> mtime = Thu Mar 08 18:36:12 PST 2012
>> pZxid = 0x27c0
>> cversion = 38
>> dataVersion = 0
>> aclVersion = 0
>> ephemeralOwner = 0x0
>> dataLength = 0
>> numChildren = 0
>>
>> This doesn't seem to happen consistently, but more often when under
>>load.

Ah, I was thrown by your comment "more often when under load". What
does that mean given your followup?

Can you do a get instead of an "ls". That will allow us to verify the
cversion is the same before/after.

Is this standalone or multiple servers? If the latter are you noticing
only on followers, or on the leader as well?

Patrick

On Fri, Mar 9, 2012 at 11:50 AM, Shelley, Ryan <[EMAIL PROTECTED]> wrote:
> There were children previously, yes, but I tried to do the delete well
> after the ephemeral children had been removed. I stopped my services (so
> ephemeral children were removed), did an "ls" which showed the children
> had been removed, try the "delete" which reported an error, then I did a
> "get" just to show the node and that it's reporting 0 children. No
> services were running against ZK while I tried the delete as I had stopped
> them all.
>
> On 3/9/12 11:43 AM, "Patrick Hunt" <[EMAIL PROTECTED]> wrote:
>
>>Timing? Perhaps by the time you get around to doing the get the child
>>has been removed, but one/more exist at the time you do the delete.
>>Notice the cversion is 38 in this case (so there were children at one
>>point).
>>
>>Patrick
>>
>>On Fri, Mar 9, 2012 at 11:40 AM, Shelley, Ryan <[EMAIL PROTECTED]>
>>wrote:
>>> Has anyone had an issue attempting to delete an empty node (zk 3.4.3):
>>>
>>> [zk: localhost:2181(CONNECTED) 252] ls /streams/foo
>>> []
>>> [zk: localhost:2181(CONNECTED) 253] delete /streams/foo
>>> Node not empty: /streams/foo
>>> [zk: localhost:2181(CONNECTED) 254] get /streams/foo
>>>
>>> cZxid = 0x245f
>>> ctime = Thu Mar 08 18:36:12 PST 2012
>>> mZxid = 0x245f
>>> mtime = Thu Mar 08 18:36:12 PST 2012
>>> pZxid = 0x27c0
>>> cversion = 38
>>> dataVersion = 0
>>> aclVersion = 0
>>> ephemeralOwner = 0x0
>>> dataLength = 0
>>> numChildren = 0
>>>
>>> This doesn't seem to happen consistently, but more often when under
>>>load.
>

Under load, meaning, if I just do manual serial requests, it doesn't
happen. If I use Jmeter and hammer my app, it may cause one of the paths
to get hung up. It's not consistent though. Sometimes it happens sometimes
not.

Next time it happens, I'll do a get/delete/get and send it in. So far,
this is on a single instance of ZK.

-Ryan

On 3/9/12 11:58 AM, "Patrick Hunt" <[EMAIL PROTECTED]> wrote:

>Ah, I was thrown by your comment "more often when under load". What
>does that mean given your followup?
>
>Can you do a get instead of an "ls". That will allow us to verify the
>cversion is the same before/after.
>
>Is this standalone or multiple servers? If the latter are you noticing
>only on followers, or on the leader as well?
>
>Patrick
>
>On Fri, Mar 9, 2012 at 11:50 AM, Shelley, Ryan <[EMAIL PROTECTED]>
>wrote:
>> There were children previously, yes, but I tried to do the delete well
>> after the ephemeral children had been removed. I stopped my services (so
>> ephemeral children were removed), did an "ls" which showed the children
>> had been removed, try the "delete" which reported an error, then I did a
>> "get" just to show the node and that it's reporting 0 children. No
>> services were running against ZK while I tried the delete as I had
>>stopped
>> them all.
>>
>> On 3/9/12 11:43 AM, "Patrick Hunt" <[EMAIL PROTECTED]> wrote:
>>
>>>Timing? Perhaps by the time you get around to doing the get the child
>>>has been removed, but one/more exist at the time you do the delete.
>>>Notice the cversion is 38 in this case (so there were children at one
>>>point).
>>>
>>>Patrick
>>>
>>>On Fri, Mar 9, 2012 at 11:40 AM, Shelley, Ryan <[EMAIL PROTECTED]>
>>>wrote:
>>>> Has anyone had an issue attempting to delete an empty node (zk 3.4.3):
>>>>
>>>> [zk: localhost:2181(CONNECTED) 252] ls /streams/foo
>>>> []
>>>> [zk: localhost:2181(CONNECTED) 253] delete /streams/foo
>>>> Node not empty: /streams/foo
>>>> [zk: localhost:2181(CONNECTED) 254] get /streams/foo
>>>>
>>>> cZxid = 0x245f
>>>> ctime = Thu Mar 08 18:36:12 PST 2012
>>>> mZxid = 0x245f
>>>> mtime = Thu Mar 08 18:36:12 PST 2012
>>>> pZxid = 0x27c0
>>>> cversion = 38
>>>> dataVersion = 0
>>>> aclVersion = 0
>>>> ephemeralOwner = 0x0
>>>> dataLength = 0
>>>> numChildren = 0
>>>>
>>>> This doesn't seem to happen consistently, but more often when under
>>>>load.
>>