Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper, mail # user - Dump all changes on ZK to file


+
kishore g 2011-09-13, 22:53
+
Benjamin Reed 2011-09-13, 23:43
+
kishore g 2011-09-13, 23:47
+
kishore g 2011-09-14, 00:40
+
kishore g 2011-09-14, 06:44
+
Patrick Hunt 2011-09-14, 17:37
Copy link to this message
-
Re: Dump all changes on ZK to file
kishore g 2011-09-14, 18:58
Will do.

https://issues.apache.org/jira/browse/ZOOKEEPER-1183

thanks
Kishore G

On Wed, Sep 14, 2011 at 10:37 AM, Patrick Hunt <[EMAIL PROTECTED]> wrote:

> Consider submitting a patch.
>
> Patrick
>
> On Tue, Sep 13, 2011 at 11:44 PM, kishore g <[EMAIL PROTECTED]> wrote:
> > Hi Ben,
> >
> > I was able to get more information by changing the LogFormatter code.
> >
> > Record txn = SerializeUtils.deserializeTxn(iab, hdr);
> >
> > Based on the txn type, I could cast it to appropriate Txn like SetDataTxn
> > and was able to get all the information needed.
> >
> > thanks,
> > Kishore G
> >
> > On Tue, Sep 13, 2011 at 5:40 PM, kishore g <[EMAIL PROTECTED]> wrote:
> >
> >> Hi Ben,
> >>
> >> This only gives me the hugh level information something like
> >>
> >> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> >> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01
> >> createSession
> >> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02
> >> setData
> >> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03
> >> createSession
> >> ...
> >> ..
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b
> >> setData
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c
> >> create
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d
> >> setData
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e
> >> setData
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f
> >> setData
> >>
> >> How can get more details about the transaction itself. I want to know
> which
> >> node was created deleted what was set etc. Is this possible ?
> >>
> >> thanks,
> >> Kishore G
> >>
> >>
> >>
> >> On Tue, Sep 13, 2011 at 4:47 PM, kishore g <[EMAIL PROTECTED]> wrote:
> >>
> >>> Thanks Ben, that's exactly what I was looking for.
> >>>
> >>> On Tue, Sep 13, 2011 at 4:43 PM, Benjamin Reed <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>>> you can do this using the logs. there is a log format utility:
> >>>> LogFormatter that will print the log in a human readable format that
> >>>> you could use to get this information. (actually you might need to
> >>>> enhance the tool a bit depending on the detail that you want.)
> >>>>
> >>>> ben
> >>>>
> >>>> On Tue, Sep 13, 2011 at 3:53 PM, kishore g <[EMAIL PROTECTED]>
> wrote:
> >>>> > Hi,
> >>>> >
> >>>> > I want to dump all changes that happen on ZK to a file.
> >>>> >
> >>>> > What I am trying to do is following
> >>>> >
> >>>> >   - Set  up some application specific structure on zookeeper.
> >>>> >   - Start a listener or something equivalent that watches on all the
> >>>> nodes.
> >>>> >   - Multiple processes are connected to Zookeeper and they make some
> >>>> >   changes on the ZK as needed
> >>>> >   - The listener will detect the changes and dump the changes to a
> file
> >>>> in
> >>>> >   some structure
> >>>> >   - Once the run is over there will be a verification tool which
> will
> >>>> go
> >>>> >   over the change log and do some verifications.
> >>>> >
> >>>> > I am not sure if there is already a tool to do this or if it is
> >>>> possible to
> >>>> > get all changes from ZK server logs.
> >>>> >
> >>>> > I am aware that if we have a separate listener we may not be able to
> >>>> get all
> >>>> > changes since I might miss changes between receiving a notification
> and
> >>>> a
> >>>> > change happening before i reset the watch. This may not be the case
> >>>> with ZK
> >>>> > logs/snapshot.
> >>>> >
> >>>> > This will not be in production but I want to use it to for testing.
> >>>> Idea is
> >>>> > to decouple verification from simulation so that all types of
> >>>> verification
> >>>> > can be done based on the change log.
> >>>> >
> >>>> > Any pointers/suggestions ?
> >>>> >
> >>>> > thanks,
> >>>> > Kishore G
> >>>> >
> >>>>
> >>>
> >>>
> >>
> >
>