Hi ZooKeepers,

I just recently had an issue about getting the a node's updated data after the watch was actually fired. Being pretty confused, I read the documentation page again and sure enough this has been stated:

"A client will see a watch event for a znode it is watching before seeing the new data that corresponds to that znode."
http://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html

Now, I'm confused about how a data watch could actually be useful... But aside from that, is this statement also true about the node's list of Children? I.e. are a node's children it's "data" too?

Because if this is the case, wouldn't the recipe for barriers hang if it gets the watch for a new node addition goes off and in:

 if (list.size() < size) {
                        mutex.wait();

 list.size() is still smaller than size?

Or is this statement about watches only true for a data on the node?

Many thanks,
Amir

On Thu, Mar 1, 2012 at 4:11 PM, Amirhossein Kiani <[EMAIL PROTECTED]> wrote:
> Hi ZooKeepers,

Hi Amir.

> I just recently had an issue about getting the a node's updated data after the watch was actually fired. Being pretty confused, I read the documentation page again and sure enough this has been stated:
>
> "A client will see a watch event for a znode it is watching before seeing the new data that corresponds to that znode."
> http://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html
>

Those docs are pretty old. Try using the current version:
http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html

> Now, I'm confused about how a data watch could actually be useful... But aside from that, is this statement also true about the node's list of Children? I.e. are a node's children it's "data" too?
>

Watches let you know something changed. Then you can take actions.
Typically this means that the data of a znode changed, or the znode
itself changed (created/deleted, children added/removed). There are
two separate calls related to your question - getData and getChildren,
they set watches on the data an child list respectively.

> Because if this is the case, wouldn't the recipe for barriers hang if it gets the watch for a new node addition goes off and in:
>
>  if (list.size() < size) {
>                        mutex.wait();
>
>  list.size() is still smaller than size?
>
> Or is this statement about watches only true for a data on the node?

No. It's true for both cases. If you set a data watch then it's
guaranteed that if you getData after being notified you will see the
updated data. Same for getChildren and it's child list.

Regards,

Patrick

Many thanks Patrick for pointing me to the new documentation. I just found the other one from Google somehow.

So what I think is happening is actually impossible: to do getData() on a node and see the OLD data. in other words, I do not need to loop on a getData() to get the actual new data after being notified about the data change.
The reason that I'm saying that is that's the behavior I'm seeing in my code, but it might be just a bug on my side...

Cheers,
Amir

On Mar 2, 2012, at 9:54 AM, Patrick Hunt wrote:

> On Thu, Mar 1, 2012 at 4:11 PM, Amirhossein Kiani <[EMAIL PROTECTED]> wrote:
>> Hi ZooKeepers,
>
> Hi Amir.
>
>> I just recently had an issue about getting the a node's updated data after the watch was actually fired. Being pretty confused, I read the documentation page again and sure enough this has been stated:
>>
>> "A client will see a watch event for a znode it is watching before seeing the new data that corresponds to that znode."
>> http://zookeeper.apache.org/doc/r3.1.2/zookeeperProgrammers.html
>>
>
> Those docs are pretty old. Try using the current version:
> http://zookeeper.apache.org/doc/current/zookeeperProgrammers.html
>
>> Now, I'm confused about how a data watch could actually be useful... But aside from that, is this statement also true about the node's list of Children? I.e. are a node's children it's "data" too?
>>
>
> Watches let you know something changed. Then you can take actions.
> Typically this means that the data of a znode changed, or the znode
> itself changed (created/deleted, children added/removed). There are
> two separate calls related to your question - getData and getChildren,
> they set watches on the data an child list respectively.
>
>> Because if this is the case, wouldn't the recipe for barriers hang if it gets the watch for a new node addition goes off and in:
>>
>>  if (list.size() < size) {
>>                        mutex.wait();
>>
>>  list.size() is still smaller than size?
>>
>> Or is this statement about watches only true for a data on the node?
>
> No. It's true for both cases. If you set a data watch then it's
> guaranteed that if you getData after being notified you will see the
> updated data. Same for getChildren and it's child list.
>
> Regards,
>
> Patrick

On Fri, Mar 2, 2012 at 11:23 AM, Amirhossein Kiani <[EMAIL PROTECTED]> wrote:
> Many thanks Patrick for pointing me to the new documentation. I just found the other one from Google somehow.
>

No problem.

> So what I think is happening is actually impossible: to do getData() on a node and see the OLD data. in other words, I do not need to loop on a getData() to get the actual new data after being notified about the data change.
> The reason that I'm saying that is that's the behavior I'm seeing in my code, but it might be just a bug on my side...

Sounds like. Keep in mind that there might be multiple changes btw the
time the notification fires and when your getData runs on the server.
Perhaps someone's changing it back? :-)

Patrick

Hmm... I tried testing the idea that if I call the getData() on updated node in the watcher's process() method I'll get the updated data.
I created a watcher class that keeps track of the values it's receiving and in my test I sequentially set 10000 values on the node. I do see that some times I'm getting the OLD VALUE.

I wonder if the way I'm setting the data or getting it is incorrect.

Here are the main files in my test:
https://gist.github.com/1961660

You can run my test by running "mvn test" in the maven project attached.

Many thanks for your help!
Amir

On Mar 2, 2012, at 11:27 AM, Patrick Hunt wrote:

> On Fri, Mar 2, 2012 at 11:23 AM, Amirhossein Kiani <[EMAIL PROTECTED]> wrote:
>> Many thanks Patrick for pointing me to the new documentation. I just found the other one from Google somehow.
>>
>
> No problem.
>
>> So what I think is happening is actually impossible: to do getData() on a node and see the OLD data. in other words, I do not need to loop on a getData() to get the actual new data after being notified about the data change.
>> The reason that I'm saying that is that's the behavior I'm seeing in my code, but it might be just a bug on my side...
>
> Sounds like. Keep in mind that there might be multiple changes btw the
> time the notification fires and when your getData runs on the server.
> Perhaps someone's changing it back? :-)
>
> Patrick

I did have an issue in my test, namely not making sure I wait for all the watches to fire. I added a check for that now: https://gist.github.com/1961660, but still the test fails.

Amir

On Mar 2, 2012, at 1:58 PM, Amirhossein Kiani wrote:

> Hmm... I tried testing the idea that if I call the getData() on updated node in the watcher's process() method I'll get the updated data.
> I created a watcher class that keeps track of the values it's receiving and in my test I sequentially set 10000 values on the node. I do see that some times I'm getting the OLD VALUE.
>
> I wonder if the way I'm setting the data or getting it is incorrect.
>
> Here are the main files in my test:
> https://gist.github.com/1961660
>
> You can run my test by running "mvn test" in the maven project attached.
> <test-get-data.tar.gz>
>
>
> Many thanks for your help!
> Amir
>
> On Mar 2, 2012, at 11:27 AM, Patrick Hunt wrote:
>
>> On Fri, Mar 2, 2012 at 11:23 AM, Amirhossein Kiani <[EMAIL PROTECTED]> wrote:
>>> Many thanks Patrick for pointing me to the new documentation. I just found the other one from Google somehow.
>>>
>>
>> No problem.
>>
>>> So what I think is happening is actually impossible: to do getData() on a node and see the OLD data. in other words, I do not need to loop on a getData() to get the actual new data after being notified about the data change.
>>> The reason that I'm saying that is that's the behavior I'm seeing in my code, but it might be just a bug on my side...
>>
>> Sounds like. Keep in mind that there might be multiple changes btw the
>> time the notification fires and when your getData runs on the server.
>> Perhaps someone's changing it back? :-)
>>
>> Patrick
>

I don't think that your test tests what you think it tests.

You won't necessarily get a notification every time a value changes.  If a
value changes and a watch is queued, you won't get another notification
until you get the data and set the new watch.  If another change happens
before you reset the watch, you will get the second changed value, not the
first.  With your test, you will see a skipped value in your table and
conclude that you saw an old value twice.  The correct inference is that
you saw every value exactly once except the skipped one.

On Fri, Mar 2, 2012 at 1:58 PM, Amirhossein Kiani <[EMAIL PROTECTED]>wrote:

> Hmm... I tried testing the idea that if I call the getData() on updated
> node in the watcher's process() method I'll get the updated data.
> I created a watcher class that keeps track of the values it's receiving
> and in my test I sequentially set 10000 values on the node. I do see that
> some times I'm getting the OLD VALUE.
>
> I wonder if the way I'm setting the data or getting it is incorrect.
>
> Here are the main files in my test:
> https://gist.github.com/1961660
>
> You can run my test by running "mvn test" in the maven project attached.
>
>
>
> Many thanks for your help!
> Amir
>
> On Mar 2, 2012, at 11:27 AM, Patrick Hunt wrote:
>
> On Fri, Mar 2, 2012 at 11:23 AM, Amirhossein Kiani <[EMAIL PROTECTED]>
> wrote:
>
> Many thanks Patrick for pointing me to the new documentation. I just found
> the other one from Google somehow.
>
>
>
> No problem.
>
> So what I think is happening is actually impossible: to do getData() on a
> node and see the OLD data. in other words, I do not need to loop on a
> getData() to get the actual new data after being notified about the data
> change.
>
> The reason that I'm saying that is that's the behavior I'm seeing in my
> code, but it might be just a bug on my side...
>
>
> Sounds like. Keep in mind that there might be multiple changes btw the
> time the notification fires and when your getData runs on the server.
> Perhaps someone's changing it back? :-)
>
> Patrick
>
>
>
>

Oh.. you are right. The value was skipped because I was getting two of some values (newer but not older)
So I should restructure my code to not make decisions that rely on locks created based on node data (unless I come up with a way to sync the state myself :D) I think I'll implement that by creating a chain of children instead.

The statement about getting every value exactly once is not correct though, because I do get some values twice (when one is skipped)

Thanks a lot!
Amir

On Mar 2, 2012, at 2:52 PM, Ted Dunning wrote:

> I don't think that your test tests what you think it tests.
>
> You won't necessarily get a notification every time a value changes.  If a
> value changes and a watch is queued, you won't get another notification
> until you get the data and set the new watch.  If another change happens
> before you reset the watch, you will get the second changed value, not the
> first.  With your test, you will see a skipped value in your table and
> conclude that you saw an old value twice.  The correct inference is that
> you saw every value exactly once except the skipped one.
>
> On Fri, Mar 2, 2012 at 1:58 PM, Amirhossein Kiani <[EMAIL PROTECTED]>wrote:
>
>> Hmm... I tried testing the idea that if I call the getData() on updated
>> node in the watcher's process() method I'll get the updated data.
>> I created a watcher class that keeps track of the values it's receiving
>> and in my test I sequentially set 10000 values on the node. I do see that
>> some times I'm getting the OLD VALUE.
>>
>> I wonder if the way I'm setting the data or getting it is incorrect.
>>
>> Here are the main files in my test:
>> https://gist.github.com/1961660
>>
>> You can run my test by running "mvn test" in the maven project attached.
>>
>>
>>
>> Many thanks for your help!
>> Amir
>>
>> On Mar 2, 2012, at 11:27 AM, Patrick Hunt wrote:
>>
>> On Fri, Mar 2, 2012 at 11:23 AM, Amirhossein Kiani <[EMAIL PROTECTED]>
>> wrote:
>>
>> Many thanks Patrick for pointing me to the new documentation. I just found
>> the other one from Google somehow.
>>
>>
>>
>> No problem.
>>
>> So what I think is happening is actually impossible: to do getData() on a
>> node and see the OLD data. in other words, I do not need to loop on a
>> getData() to get the actual new data after being notified about the data
>> change.
>>
>> The reason that I'm saying that is that's the behavior I'm seeing in my
>> code, but it might be just a bug on my side...
>>
>>
>> Sounds like. Keep in mind that there might be multiple changes btw the
>> time the notification fires and when your getData runs on the server.
>> Perhaps someone's changing it back? :-)
>>
>> Patrick
>>
>>
>>
>>

On Fri, Mar 2, 2012 at 3:33 PM, Amirhossein Kiani <[EMAIL PROTECTED]>wrote:

> Oh.. you are right. The value was skipped because I was getting two of
> some values (newer but not older)
> So I should restructure my code to not make decisions that rely on locks
> created based on node data (unless I come up with a way to sync the state
> myself :D) I think I'll implement that by creating a chain of children
> instead.
>

You can atomically update data using ZK pretty easily.  That may be all you
need.  Alternately, the existence of a file serves as a lock.

But generally I recommend against using locks as such in distributed
systems.  There are usually better structures.

The statement about getting every value exactly once is not correct though,
> because I do get some values twice (when one is skipped)
>

I think you shouldn't.  When a value is skipped, it is because two changes
caused one notification.  Watches are removed as soon as they are
triggered.  If the value was n before the change and the change to n+1
triggers your watch, then the get in the watcher will either get n+1 or
possibly n+2 if the changes are very fast. In the first case, you will get
n+1 and set the watch before the next change.  In the second, you will get
n+2 and set the watch *after* the change to n+2.  The notification after
this will give you n+3 or n+4 and you should never see n+2 again.