Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> requiring zookeeper 3.4.x in 0.96


Copy link to this message
-
Re: requiring zookeeper 3.4.x in 0.96
-------------------
Jesse Yates
240-888-2200
@jesse_yates
jyates.github.com
On Wed, Apr 18, 2012 at 3:34 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> On Wed, Apr 18, 2012 at 3:29 PM, Jesse Yates <[EMAIL PROTECTED]>
> wrote:
> > Its makes dealing with a lot of the zk operations easier to reason about.
> > Consider the 'create with parents' or the 'delete recursively' use cases
> -
> > non-transactional guarantees means we have a lot of extra logic for
> > checking under parents on creation, synchronous operations requiring
> extra
> > checking/notification etc; in short, its a lot of logic for things that
> > aren't, on a high-level, all that complicated (and are thought of
> > transactionally).
>
> But how much do we really do these things? Isn't it only when we
> bootstrap a new cluster?
>

Depends on the operation - the replication code has a lot of reliance of
moving zk nodes around and the hfile backup stuff that is coming also does
dynamic znode allocation.
>
>
>
 >
> > These are the corner cases that are harder to find. For instance, what if
> > the parent is there, but the child has been put there - I need to leave a
> > watch and then do an update when children changed, and then read in all
> the
> > children, which could be actually more than I'm looking for, which leads
> to
> > a bunch of extra IO. If you can be sure that this goes away, we can get
> > much simpler code - notify for the node and then read the children. If
> the
> > children aren't there, they won't show up. this is just one possible
> > example, but others can also be constructed.
>
> You mean in the middle of a recursive op?
>

Yeah, exactly. Its an annoying use case and one easily messed up.
>
> >
> > It also can enable some really nice, new use cases. For instance, I've
> > discussed doing a Paxos-like mechanism that is far more easily enabled by
> > having transactional read-modify-write semantics for things like dynamic
> > table configuration (not proposing this as a jira (yet), and don't want
> to
> > get into if its a good idea, but rather just as an example for possible
> new
> > uses).
>
> ZK already has version-checking setData() calls which is enough for
> optimistic concurrency... I don't get the "paxos-like mechanism" here
> when ZK is already giving us consensus and atomic CAS.
>

Oh yeah, you are right. A little confused there on where the ZK version
stuff comes from (since you don't actually set the version, but are just
given it by zk)
>
> -Todd
>
> >
> > On Wed, Apr 18, 2012 at 2:49 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:
> >
> >> ZK 3.4.x still isn't marked "stable", so I'm a little hesitant to move
> >> to requiring it just yet.
> >>
> >> What tangible benefit do we gain from the "transactional" methods?
> >>
> >> -Todd
> >>
> >> On Wed, Apr 18, 2012 at 1:48 PM, lars hofhansl <[EMAIL PROTECTED]>
> >> wrote:
> >> > +1
> >> >
> >> >
> >> >
> >> > ----- Original Message -----
> >> > From: Jesse Yates <[EMAIL PROTECTED]>
> >> > To: [EMAIL PROTECTED]
> >> > Cc:
> >> > Sent: Wednesday, April 18, 2012 1:33 PM
> >> > Subject: requiring zookeeper 3.4.x in 0.96
> >> >
> >> > Hi all,
> >> >
> >> > I was working on some patches to make ZK a bit more sane using the
> >> > transaction operations recently added (in jira:
> >> > https://issues.apache.org/jira/browse/ZOOKEEPER-965
> >> > <
> >>
> http://zookeeper.apache.org/doc/r3.4.3/api/org/apache/zookeeper/ZooKeeper.html#transaction%28%29
> >> >
> >> > and in the api here:
> >> >
> >>
> http://zookeeper.apache.org/doc/r3.4.3/api/org/apache/zookeeper/ZooKeeper.html#transaction%28%29
> >> ).
> >> > This takes care of a lot of race conditions, etc. that are inherently
> >> > present in zk.
> >> >
> >> > Currently we (afaik) are not requiring ZooKeeper v.3.4.x in production
> >> and
> >> > instead allow users to run down to at least 3.3.x with the hbase's
>  3.4.x
> >> > client. Adding in transaction support will break this cross-version
> >> > functionality without a way of falling back as ZK doesn't (afaik)