Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Dump all changes on ZK to file


+
kishore g 2011-09-13, 22:53
+
Benjamin Reed 2011-09-13, 23:43
+
kishore g 2011-09-13, 23:47
+
kishore g 2011-09-14, 00:40
Copy link to this message
-
Re: Dump all changes on ZK to file
Hi Ben,

I was able to get more information by changing the LogFormatter code.

Record txn = SerializeUtils.deserializeTxn(iab, hdr);

Based on the txn type, I could cast it to appropriate Txn like SetDataTxn
and was able to get all the information needed.

thanks,
Kishore G

On Tue, Sep 13, 2011 at 5:40 PM, kishore g <[EMAIL PROTECTED]> wrote:

> Hi Ben,
>
> This only gives me the hugh level information something like
>
> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01
> createSession
> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02
> setData
> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03
> createSession
> ...
> ..
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b
> setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c
> create
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d
> setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e
> setData
> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f
> setData
>
> How can get more details about the transaction itself. I want to know which
> node was created deleted what was set etc. Is this possible ?
>
> thanks,
> Kishore G
>
>
>
> On Tue, Sep 13, 2011 at 4:47 PM, kishore g <[EMAIL PROTECTED]> wrote:
>
>> Thanks Ben, that's exactly what I was looking for.
>>
>> On Tue, Sep 13, 2011 at 4:43 PM, Benjamin Reed <[EMAIL PROTECTED]> wrote:
>>
>>> you can do this using the logs. there is a log format utility:
>>> LogFormatter that will print the log in a human readable format that
>>> you could use to get this information. (actually you might need to
>>> enhance the tool a bit depending on the detail that you want.)
>>>
>>> ben
>>>
>>> On Tue, Sep 13, 2011 at 3:53 PM, kishore g <[EMAIL PROTECTED]> wrote:
>>> > Hi,
>>> >
>>> > I want to dump all changes that happen on ZK to a file.
>>> >
>>> > What I am trying to do is following
>>> >
>>> >   - Set  up some application specific structure on zookeeper.
>>> >   - Start a listener or something equivalent that watches on all the
>>> nodes.
>>> >   - Multiple processes are connected to Zookeeper and they make some
>>> >   changes on the ZK as needed
>>> >   - The listener will detect the changes and dump the changes to a file
>>> in
>>> >   some structure
>>> >   - Once the run is over there will be a verification tool which will
>>> go
>>> >   over the change log and do some verifications.
>>> >
>>> > I am not sure if there is already a tool to do this or if it is
>>> possible to
>>> > get all changes from ZK server logs.
>>> >
>>> > I am aware that if we have a separate listener we may not be able to
>>> get all
>>> > changes since I might miss changes between receiving a notification and
>>> a
>>> > change happening before i reset the watch. This may not be the case
>>> with ZK
>>> > logs/snapshot.
>>> >
>>> > This will not be in production but I want to use it to for testing.
>>> Idea is
>>> > to decouple verification from simulation so that all types of
>>> verification
>>> > can be done based on the change log.
>>> >
>>> > Any pointers/suggestions ?
>>> >
>>> > thanks,
>>> > Kishore G
>>> >
>>>
>>
>>
>
+
Patrick Hunt 2011-09-14, 17:37
+
kishore g 2011-09-14, 18:58