Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Zookeeper >> mail # user >> Dump all changes on ZK to file


+
kishore g 2011-09-13, 22:53
+
Benjamin Reed 2011-09-13, 23:43
+
kishore g 2011-09-13, 23:47
+
kishore g 2011-09-14, 00:40
+
kishore g 2011-09-14, 06:44
+
Patrick Hunt 2011-09-14, 17:37
Copy link to this message
-
Re: Dump all changes on ZK to file
Will do.

https://issues.apache.org/jira/browse/ZOOKEEPER-1183

thanks
Kishore G

On Wed, Sep 14, 2011 at 10:37 AM, Patrick Hunt <[EMAIL PROTECTED]> wrote:

> Consider submitting a patch.
>
> Patrick
>
> On Tue, Sep 13, 2011 at 11:44 PM, kishore g <[EMAIL PROTECTED]> wrote:
> > Hi Ben,
> >
> > I was able to get more information by changing the LogFormatter code.
> >
> > Record txn = SerializeUtils.deserializeTxn(iab, hdr);
> >
> > Based on the txn type, I could cast it to appropriate Txn like SetDataTxn
> > and was able to get all the information needed.
> >
> > thanks,
> > Kishore G
> >
> > On Tue, Sep 13, 2011 at 5:40 PM, kishore g <[EMAIL PROTECTED]> wrote:
> >
> >> Hi Ben,
> >>
> >> This only gives me the hugh level information something like
> >>
> >> ZooKeeper Transactional Log File with dbid 0 txnlog format version 2
> >> 8/15/11 1:55:36 PM PDT session 0x131cf1a236f0014 cxid 0x0 zxid 0xf01
> >> createSession
> >> 8/15/11 1:55:57 PM PDT session 0x131cf1a236f0000 cxid 0x55f zxid 0xf02
> >> setData
> >> 8/15/11 1:56:00 PM PDT session 0x131cf1a236f0015 cxid 0x0 zxid 0xf03
> >> createSession
> >> ...
> >> ..
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001c cxid 0x36 zxid 0xf6b
> >> setData
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f0021 cxid 0xa1 zxid 0xf6c
> >> create
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001b cxid 0x3e zxid 0xf6d
> >> setData
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001e cxid 0x3e zxid 0xf6e
> >> setData
> >> 8/15/11 2:00:33 PM PDT session 0x131cf1a236f001d cxid 0x41 zxid 0xf6f
> >> setData
> >>
> >> How can get more details about the transaction itself. I want to know
> which
> >> node was created deleted what was set etc. Is this possible ?
> >>
> >> thanks,
> >> Kishore G
> >>
> >>
> >>
> >> On Tue, Sep 13, 2011 at 4:47 PM, kishore g <[EMAIL PROTECTED]> wrote:
> >>
> >>> Thanks Ben, that's exactly what I was looking for.
> >>>
> >>> On Tue, Sep 13, 2011 at 4:43 PM, Benjamin Reed <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>>> you can do this using the logs. there is a log format utility:
> >>>> LogFormatter that will print the log in a human readable format that
> >>>> you could use to get this information. (actually you might need to
> >>>> enhance the tool a bit depending on the detail that you want.)
> >>>>
> >>>> ben
> >>>>
> >>>> On Tue, Sep 13, 2011 at 3:53 PM, kishore g <[EMAIL PROTECTED]>
> wrote:
> >>>> > Hi,
> >>>> >
> >>>> > I want to dump all changes that happen on ZK to a file.
> >>>> >
> >>>> > What I am trying to do is following
> >>>> >
> >>>> >   - Set  up some application specific structure on zookeeper.
> >>>> >   - Start a listener or something equivalent that watches on all the
> >>>> nodes.
> >>>> >   - Multiple processes are connected to Zookeeper and they make some
> >>>> >   changes on the ZK as needed
> >>>> >   - The listener will detect the changes and dump the changes to a
> file
> >>>> in
> >>>> >   some structure
> >>>> >   - Once the run is over there will be a verification tool which
> will
> >>>> go
> >>>> >   over the change log and do some verifications.
> >>>> >
> >>>> > I am not sure if there is already a tool to do this or if it is
> >>>> possible to
> >>>> > get all changes from ZK server logs.
> >>>> >
> >>>> > I am aware that if we have a separate listener we may not be able to
> >>>> get all
> >>>> > changes since I might miss changes between receiving a notification
> and
> >>>> a
> >>>> > change happening before i reset the watch. This may not be the case
> >>>> with ZK
> >>>> > logs/snapshot.
> >>>> >
> >>>> > This will not be in production but I want to use it to for testing.
> >>>> Idea is
> >>>> > to decouple verification from simulation so that all types of
> >>>> verification
> >>>> > can be done based on the change log.
> >>>> >
> >>>> > Any pointers/suggestions ?
> >>>> >
> >>>> > thanks,
> >>>> > Kishore G
> >>>> >
> >>>>
> >>>
> >>>
> >>
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB