Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> requiring zookeeper 3.4.x in 0.96


Copy link to this message
-
Re: requiring zookeeper 3.4.x in 0.96
-------------------
Jesse Yates
240-888-2200
@jesse_yates
jyates.github.com
On Wed, Apr 18, 2012 at 3:34 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> On Wed, Apr 18, 2012 at 3:29 PM, Jesse Yates <[EMAIL PROTECTED]>
> wrote:
> > Its makes dealing with a lot of the zk operations easier to reason about.
> > Consider the 'create with parents' or the 'delete recursively' use cases
> -
> > non-transactional guarantees means we have a lot of extra logic for
> > checking under parents on creation, synchronous operations requiring
> extra
> > checking/notification etc; in short, its a lot of logic for things that
> > aren't, on a high-level, all that complicated (and are thought of
> > transactionally).
>
> But how much do we really do these things? Isn't it only when we
> bootstrap a new cluster?
>

Depends on the operation - the replication code has a lot of reliance of
moving zk nodes around and the hfile backup stuff that is coming also does
dynamic znode allocation.
>
>
>
 >
> > These are the corner cases that are harder to find. For instance, what if
> > the parent is there, but the child has been put there - I need to leave a
> > watch and then do an update when children changed, and then read in all
> the
> > children, which could be actually more than I'm looking for, which leads
> to
> > a bunch of extra IO. If you can be sure that this goes away, we can get
> > much simpler code - notify for the node and then read the children. If
> the
> > children aren't there, they won't show up. this is just one possible
> > example, but others can also be constructed.
>
> You mean in the middle of a recursive op?
>

Yeah, exactly. Its an annoying use case and one easily messed up.
>
> >
> > It also can enable some really nice, new use cases. For instance, I've
> > discussed doing a Paxos-like mechanism that is far more easily enabled by
> > having transactional read-modify-write semantics for things like dynamic
> > table configuration (not proposing this as a jira (yet), and don't want
> to
> > get into if its a good idea, but rather just as an example for possible
> new
> > uses).
>
> ZK already has version-checking setData() calls which is enough for
> optimistic concurrency... I don't get the "paxos-like mechanism" here
> when ZK is already giving us consensus and atomic CAS.
>

Oh yeah, you are right. A little confused there on where the ZK version
stuff comes from (since you don't actually set the version, but are just
given it by zk)
>
> -Todd
>
> >
> > On Wed, Apr 18, 2012 at 2:49 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> >
> >> ZK 3.4.x still isn't marked "stable", so I'm a little hesitant to move
> >> to requiring it just yet.
> >>
> >> What tangible benefit do we gain from the "transactional" methods?
> >>
> >> -Todd
> >>
> >> On Wed, Apr 18, 2012 at 1:48 PM, lars hofhansl <[EMAIL PROTECTED]>
> >> wrote:
> >> > +1
> >> >
> >> >
> >> >
> >> > ----- Original Message -----
> >> > From: Jesse Yates <[EMAIL PROTECTED]>
> >> > To: [EMAIL PROTECTED]
> >> > Cc:
> >> > Sent: Wednesday, April 18, 2012 1:33 PM
> >> > Subject: requiring zookeeper 3.4.x in 0.96
> >> >
> >> > Hi all,
> >> >
> >> > I was working on some patches to make ZK a bit more sane using the
> >> > transaction operations recently added (in jira:
> >> > https://issues.apache.org/jira/browse/ZOOKEEPER-965
> >> > <
> >>
> http://zookeeper.apache.org/doc/r3.4.3/api/org/apache/zookeeper/ZooKeeper.html#transaction%28%29
> >> >
> >> > and in the api here:
> >> >
> >>
> http://zookeeper.apache.org/doc/r3.4.3/api/org/apache/zookeeper/ZooKeeper.html#transaction%28%29
> >> ).
> >> > This takes care of a lot of race conditions, etc. that are inherently
> >> > present in zk.
> >> >
> >> > Currently we (afaik) are not requiring ZooKeeper v.3.4.x in production
> >> and
> >> > instead allow users to run down to at least 3.3.x with the hbase's
>  3.4.x
> >> > client. Adding in transaction support will break this cross-version
> >> > functionality without a way of falling back as ZK doesn't (afaik)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB