Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper >> mail # dev >> Re: Extracting Zab from Zookeeper


Copy link to this message
-
Re: Extracting Zab from Zookeeper
flavio and ted are correct. zab and zookeeper are tightly coupled, and
as flavio points out, zab provides a bit more than just atomic broadcast.

i think it would be a good thing to do for three reasons:

1) it would make testing, benchmarking, and fixing that layer
(especially for ZOOKEEPER-22) much easier.
2) we could try different backends easier. (i'd love to try using
bookkeeper for example.)
3) although the zab interface must be richer than normal atomic
broadcast (access to past transactions, for example), i think in
practice applications that do primary/backup can benefit from this
richer interface.

ben

On 01/28/2011 07:03 AM, Ted Dunning wrote:
> I think that what Flavio was saying is that it is like pulling a string on a
> sweater.  Almost any application that wants ZAB is probably going to need
> the broadcast and they the broadcast, they will want to have logging.  And
> transactions.  And so on.
>
> Moreover, some of these features are not necessarily encoded in ZK in a way
> that you can point to.  Instead they are enabled by the way that several
> functional units are glued together.
>
> On Thu, Jan 27, 2011 at 11:24 PM, André Oriani<[EMAIL PROTECTED]>  wrote:
>
>> Sorry  for the very late reply but I am having a hard time lately. Flavio,
>> I am not sure if I understood your point. I know that Zookeeper and Zab are
>> very tied. The snapshot for Zab is the serialized version of ZooKepper's
>> tree  and the proposal id, zxid, is intensively used by Zookeeper
>> application. Are you meaning that it will not be possible to get two very
>> distinct layers : Zab atomic broadcast protocol and ZooKeper application?
>> And I will not be able to use the atomic broadcast layer in isolate ?
>>
>>
>> Thanks and Regards,
>> André
>>
>> On Sat, Jan 22, 2011 at 07:26, Flavio Junqueira<[EMAIL PROTECTED]>  wrote:
>>
>>> Not only it won't be trivial, but I don't have it completely clear that if
>>> we separate Zab and ZooKeeper you will have exactly what you need. In our
>>> previous discussions about this issue, we ended up concluding that we would
>>> need a richer interface than simply abcast and abdeliver. For example, one
>>> important aspect currently is stable storage for logging and checkpointing.
>>> To separate the ZooKeeper application and Zab we would have to decide which
>>> side would manage it, but both sides would need access to to it some way.
>>>
>>> It sounds like a good idea to think carefully if a separate Zab would
>>> really satisfy your requirements before spending time on the
>>> implementation.
>>>
>>> -Flavio
>>>
>>> On Jan 22, 2011, at 8:43 AM, André Oriani wrote:
>>>
>>> If I do not reply soon, do not think I gave up. My employer is keeping me
>>> busy lately :)
>>>
>>> I will try take a look at the patch of jira
>>>
>>> Regards,
>>> André
>>>
>>> On Thu, Jan 20, 2011 at 09:45, Patrick Hunt<[EMAIL PROTECTED]>  wrote:
>>>
>>> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
>>>
>>> finished work product from the original effort. I suspect that this
>>>
>>> would be a great starting point, most of the work being to "port"
>>>
>>> those changes onto the latest trunk. Ben what do you think? Was the
>>>
>>> original approach solid? Seems like a great starting point.
>>>
>>>
>>> Patrick
>>>
>>>
>>> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani<[EMAIL PROTECTED]>  wrote:
>>>
>>> Sure, I am interested in. What should I do besides reading the papers,
>>>
>>> downloading the code compiling and doing some code walkthrough ?
>>>
>>>
>>>
>>> Tks,
>>>
>>> André
>>>
>>>
>>> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed<[EMAIL PROTECTED]>
>>>
>>> wrote:
>>>
>>>
>>> it's funny, i was just thinking about this yesterday.
>>>
>>>
>>> no one is working on it, so it is still open.
>>>
>>>
>>> it is a non-trivial piece of work, but i'd be willing to give guidance
>>>
>>> if
>>>
>>> you are interested in it.
>>>
>>>
>>> ben
>>>
>>>
>>> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB