Benjamin Reed
2011-01-19, 19:32
André Oriani
2011-01-20, 16:35
Patrick Hunt
2011-01-20, 17:45
André Oriani
2011-01-22, 07:43
Flavio Junqueira
2011-01-22, 15:26
André Oriani
2011-01-28, 07:24
Ted Dunning
2011-01-28, 15:03
Benjamin Reed
2011-01-28, 18:27
Andrew Purtell
2011-01-31, 00:56
Mahadev Konar
2011-01-31, 03:21
Andrew Purtell
2011-02-01, 16:38
Flavio Junqueira
2011-02-02, 10:14
Andrew Purtell
2011-02-03, 18:52
André Oriani
2011-06-11, 02:36
-
Re: Extracting Zab from ZookeeperBenjamin Reed 2011-01-19, 19:32
it's funny, i was just thinking about this yesterday.
no one is working on it, so it is still open.
it is a non-trivial piece of work, but i'd be willing to give guidance
if you are interested in it.
ben
ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
On 01/18/2011 11:24 PM, Andr� Oriani wrote:
> Hi,
>
> I need to implement a replication protocol and therefore I need total order
> broadcast primitive. I was thinking of using Zab. But I saw that the
> broadcast protocol is very intermixed with Zookeeper's tree. I heard of
> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution is
> still needed I would be glad to do it if someone could mentor me on the
> task.
>
>
> Thanks,
> Andr� Oriani
>
>
> --
> Andr� Oriani
> MSc candidate at Computing Institute
> State University of Campinas - Brazil
> (temporarily at Sunnyvale, CA)
no one is working on it, so it is still open.
it is a non-trivial piece of work, but i'd be willing to give guidance
if you are interested in it.
ben
ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
On 01/18/2011 11:24 PM, Andr� Oriani wrote:
> Hi,
>
> I need to implement a replication protocol and therefore I need total order
> broadcast primitive. I was thinking of using Zab. But I saw that the
> broadcast protocol is very intermixed with Zookeeper's tree. I heard of
> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution is
> still needed I would be glad to do it if someone could mentor me on the
> task.
>
>
> Thanks,
> Andr� Oriani
>
>
> --
> Andr� Oriani
> MSc candidate at Computing Institute
> State University of Campinas - Brazil
> (temporarily at Sunnyvale, CA)
-
Re: Extracting Zab from ZookeeperAndré Oriani 2011-01-20, 16:35
Sure, I am interested in. What should I do besides reading the papers,
downloading the code compiling and doing some code walkthrough ?
Tks,
André
On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]> wrote:
> it's funny, i was just thinking about this yesterday.
>
> no one is working on it, so it is still open.
>
> it is a non-trivial piece of work, but i'd be willing to give guidance if
> you are interested in it.
>
> ben
>
> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
>
>
> On 01/18/2011 11:24 PM, André Oriani wrote:
>
>> Hi,
>>
>> I need to implement a replication protocol and therefore I need total
>> order
>> broadcast primitive. I was thinking of using Zab. But I saw that the
>> broadcast protocol is very intermixed with Zookeeper's tree. I heard of
>> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution is
>> still needed I would be glad to do it if someone could mentor me on the
>> task.
>>
>>
>> Thanks,
>> André Oriani
>>
>>
>> --
>> André Oriani
>> MSc candidate at Computing Institute
>> State University of Campinas - Brazil
>> (temporarily at Sunnyvale, CA)
>>
>
>
downloading the code compiling and doing some code walkthrough ?
Tks,
André
On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]> wrote:
> it's funny, i was just thinking about this yesterday.
>
> no one is working on it, so it is still open.
>
> it is a non-trivial piece of work, but i'd be willing to give guidance if
> you are interested in it.
>
> ben
>
> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
>
>
> On 01/18/2011 11:24 PM, André Oriani wrote:
>
>> Hi,
>>
>> I need to implement a replication protocol and therefore I need total
>> order
>> broadcast primitive. I was thinking of using Zab. But I saw that the
>> broadcast protocol is very intermixed with Zookeeper's tree. I heard of
>> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution is
>> still needed I would be glad to do it if someone could mentor me on the
>> task.
>>
>>
>> Thanks,
>> André Oriani
>>
>>
>> --
>> André Oriani
>> MSc candidate at Computing Institute
>> State University of Campinas - Brazil
>> (temporarily at Sunnyvale, CA)
>>
>
>
-
Re: Extracting Zab from ZookeeperPatrick Hunt 2011-01-20, 17:45
It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
finished work product from the original effort. I suspect that this
would be a great starting point, most of the work being to "port"
those changes onto the latest trunk. Ben what do you think? Was the
original approach solid? Seems like a great starting point.
Patrick
On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]> wrote:
> Sure, I am interested in. What should I do besides reading the papers,
> downloading the code compiling and doing some code walkthrough ?
>
>
> Tks,
> André
>
> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]> wrote:
>
>> it's funny, i was just thinking about this yesterday.
>>
>> no one is working on it, so it is still open.
>>
>> it is a non-trivial piece of work, but i'd be willing to give guidance if
>> you are interested in it.
>>
>> ben
>>
>> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
>>
>>
>> On 01/18/2011 11:24 PM, André Oriani wrote:
>>
>>> Hi,
>>>
>>> I need to implement a replication protocol and therefore I need total
>>> order
>>> broadcast primitive. I was thinking of using Zab. But I saw that the
>>> broadcast protocol is very intermixed with Zookeeper's tree. I heard of
>>> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution is
>>> still needed I would be glad to do it if someone could mentor me on the
>>> task.
>>>
>>>
>>> Thanks,
>>> André Oriani
>>>
>>>
>>> --
>>> André Oriani
>>> MSc candidate at Computing Institute
>>> State University of Campinas - Brazil
>>> (temporarily at Sunnyvale, CA)
>>>
>>
>>
>
finished work product from the original effort. I suspect that this
would be a great starting point, most of the work being to "port"
those changes onto the latest trunk. Ben what do you think? Was the
original approach solid? Seems like a great starting point.
Patrick
On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]> wrote:
> Sure, I am interested in. What should I do besides reading the papers,
> downloading the code compiling and doing some code walkthrough ?
>
>
> Tks,
> André
>
> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]> wrote:
>
>> it's funny, i was just thinking about this yesterday.
>>
>> no one is working on it, so it is still open.
>>
>> it is a non-trivial piece of work, but i'd be willing to give guidance if
>> you are interested in it.
>>
>> ben
>>
>> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
>>
>>
>> On 01/18/2011 11:24 PM, André Oriani wrote:
>>
>>> Hi,
>>>
>>> I need to implement a replication protocol and therefore I need total
>>> order
>>> broadcast primitive. I was thinking of using Zab. But I saw that the
>>> broadcast protocol is very intermixed with Zookeeper's tree. I heard of
>>> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution is
>>> still needed I would be glad to do it if someone could mentor me on the
>>> task.
>>>
>>>
>>> Thanks,
>>> André Oriani
>>>
>>>
>>> --
>>> André Oriani
>>> MSc candidate at Computing Institute
>>> State University of Campinas - Brazil
>>> (temporarily at Sunnyvale, CA)
>>>
>>
>>
>
-
Re: Extracting Zab from ZookeeperAndré Oriani 2011-01-22, 07:43
If I do not reply soon, do not think I gave up. My employer is keeping me
busy lately :)
I will try take a look at the patch of jira
Regards,
André
On Thu, Jan 20, 2011 at 09:45, Patrick Hunt <[EMAIL PROTECTED]> wrote:
> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
> finished work product from the original effort. I suspect that this
> would be a great starting point, most of the work being to "port"
> those changes onto the latest trunk. Ben what do you think? Was the
> original approach solid? Seems like a great starting point.
>
> Patrick
>
> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]> wrote:
> > Sure, I am interested in. What should I do besides reading the papers,
> > downloading the code compiling and doing some code walkthrough ?
> >
> >
> > Tks,
> > André
> >
> > On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]>
> wrote:
> >
> >> it's funny, i was just thinking about this yesterday.
> >>
> >> no one is working on it, so it is still open.
> >>
> >> it is a non-trivial piece of work, but i'd be willing to give guidance
> if
> >> you are interested in it.
> >>
> >> ben
> >>
> >> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
> >>
> >>
> >> On 01/18/2011 11:24 PM, André Oriani wrote:
> >>
> >>> Hi,
> >>>
> >>> I need to implement a replication protocol and therefore I need total
> >>> order
> >>> broadcast primitive. I was thinking of using Zab. But I saw that the
> >>> broadcast protocol is very intermixed with Zookeeper's tree. I heard
> of
> >>> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution
> is
> >>> still needed I would be glad to do it if someone could mentor me on
> the
> >>> task.
> >>>
> >>>
> >>> Thanks,
> >>> André Oriani
> >>>
> >>>
> >>> --
> >>> André Oriani
> >>> MSc candidate at Computing Institute
> >>> State University of Campinas - Brazil
> >>> (temporarily at Sunnyvale, CA)
> >>>
> >>
> >>
> >
>
busy lately :)
I will try take a look at the patch of jira
Regards,
André
On Thu, Jan 20, 2011 at 09:45, Patrick Hunt <[EMAIL PROTECTED]> wrote:
> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
> finished work product from the original effort. I suspect that this
> would be a great starting point, most of the work being to "port"
> those changes onto the latest trunk. Ben what do you think? Was the
> original approach solid? Seems like a great starting point.
>
> Patrick
>
> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]> wrote:
> > Sure, I am interested in. What should I do besides reading the papers,
> > downloading the code compiling and doing some code walkthrough ?
> >
> >
> > Tks,
> > André
> >
> > On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]>
> wrote:
> >
> >> it's funny, i was just thinking about this yesterday.
> >>
> >> no one is working on it, so it is still open.
> >>
> >> it is a non-trivial piece of work, but i'd be willing to give guidance
> if
> >> you are interested in it.
> >>
> >> ben
> >>
> >> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
> >>
> >>
> >> On 01/18/2011 11:24 PM, André Oriani wrote:
> >>
> >>> Hi,
> >>>
> >>> I need to implement a replication protocol and therefore I need total
> >>> order
> >>> broadcast primitive. I was thinking of using Zab. But I saw that the
> >>> broadcast protocol is very intermixed with Zookeeper's tree. I heard
> of
> >>> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution
> is
> >>> still needed I would be glad to do it if someone could mentor me on
> the
> >>> task.
> >>>
> >>>
> >>> Thanks,
> >>> André Oriani
> >>>
> >>>
> >>> --
> >>> André Oriani
> >>> MSc candidate at Computing Institute
> >>> State University of Campinas - Brazil
> >>> (temporarily at Sunnyvale, CA)
> >>>
> >>
> >>
> >
>
-
Re: Extracting Zab from ZookeeperFlavio Junqueira 2011-01-22, 15:26
Not only it won't be trivial, but I don't have it completely clear
that if we separate Zab and ZooKeeper you will have exactly what you
need. In our previous discussions about this issue, we ended up
concluding that we would need a richer interface than simply abcast
and abdeliver. For example, one important aspect currently is stable
storage for logging and checkpointing. To separate the ZooKeeper
application and Zab we would have to decide which side would manage
it, but both sides would need access to to it some way.
It sounds like a good idea to think carefully if a separate Zab would
really satisfy your requirements before spending time on the
implementation.
-Flavio
On Jan 22, 2011, at 8:43 AM, André Oriani wrote:
> If I do not reply soon, do not think I gave up. My employer is
> keeping me
> busy lately :)
>
> I will try take a look at the patch of jira
>
> Regards,
> André
>
> On Thu, Jan 20, 2011 at 09:45, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>
>> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show
>> the
>> finished work product from the original effort. I suspect that this
>> would be a great starting point, most of the work being to "port"
>> those changes onto the latest trunk. Ben what do you think? Was the
>> original approach solid? Seems like a great starting point.
>>
>> Patrick
>>
>> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]>
>> wrote:
>>> Sure, I am interested in. What should I do besides reading the
>>> papers,
>>> downloading the code compiling and doing some code walkthrough ?
>>>
>>>
>>> Tks,
>>> André
>>>
>>> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]>
>> wrote:
>>>
>>>> it's funny, i was just thinking about this yesterday.
>>>>
>>>> no one is working on it, so it is still open.
>>>>
>>>> it is a non-trivial piece of work, but i'd be willing to give
>>>> guidance
>> if
>>>> you are interested in it.
>>>>
>>>> ben
>>>>
>>>> ps - btw, this is definitely a [EMAIL PROTECTED]
>>>> discussion :)
>>>>
>>>>
>>>> On 01/18/2011 11:24 PM, André Oriani wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I need to implement a replication protocol and therefore I need
>>>>> total
>>>>> order
>>>>> broadcast primitive. I was thinking of using Zab. But I saw that
>>>>> the
>>>>> broadcast protocol is very intermixed with Zookeeper's tree. I
>>>>> heard
>> of
>>>>> ZOOKEEPER-30 . What happen of it ? It still open. If code
>>>>> contribution
>> is
>>>>> still needed I would be glad to do it if someone could mentor
>>>>> me on
>> the
>>>>> task.
>>>>>
>>>>>
>>>>> Thanks,
>>>>> André Oriani
>>>>>
>>>>>
>>>>> --
>>>>> André Oriani
>>>>> MSc candidate at Computing Institute
>>>>> State University of Campinas - Brazil
>>>>> (temporarily at Sunnyvale, CA)
>>>>>
>>>>
>>>>
>>>
>>
flavio
junqueira
research scientist
[EMAIL PROTECTED]
direct +34 93-183-8828
avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300 fax (408) 349 3301
that if we separate Zab and ZooKeeper you will have exactly what you
need. In our previous discussions about this issue, we ended up
concluding that we would need a richer interface than simply abcast
and abdeliver. For example, one important aspect currently is stable
storage for logging and checkpointing. To separate the ZooKeeper
application and Zab we would have to decide which side would manage
it, but both sides would need access to to it some way.
It sounds like a good idea to think carefully if a separate Zab would
really satisfy your requirements before spending time on the
implementation.
-Flavio
On Jan 22, 2011, at 8:43 AM, André Oriani wrote:
> If I do not reply soon, do not think I gave up. My employer is
> keeping me
> busy lately :)
>
> I will try take a look at the patch of jira
>
> Regards,
> André
>
> On Thu, Jan 20, 2011 at 09:45, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>
>> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show
>> the
>> finished work product from the original effort. I suspect that this
>> would be a great starting point, most of the work being to "port"
>> those changes onto the latest trunk. Ben what do you think? Was the
>> original approach solid? Seems like a great starting point.
>>
>> Patrick
>>
>> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]>
>> wrote:
>>> Sure, I am interested in. What should I do besides reading the
>>> papers,
>>> downloading the code compiling and doing some code walkthrough ?
>>>
>>>
>>> Tks,
>>> André
>>>
>>> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]>
>> wrote:
>>>
>>>> it's funny, i was just thinking about this yesterday.
>>>>
>>>> no one is working on it, so it is still open.
>>>>
>>>> it is a non-trivial piece of work, but i'd be willing to give
>>>> guidance
>> if
>>>> you are interested in it.
>>>>
>>>> ben
>>>>
>>>> ps - btw, this is definitely a [EMAIL PROTECTED]
>>>> discussion :)
>>>>
>>>>
>>>> On 01/18/2011 11:24 PM, André Oriani wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I need to implement a replication protocol and therefore I need
>>>>> total
>>>>> order
>>>>> broadcast primitive. I was thinking of using Zab. But I saw that
>>>>> the
>>>>> broadcast protocol is very intermixed with Zookeeper's tree. I
>>>>> heard
>> of
>>>>> ZOOKEEPER-30 . What happen of it ? It still open. If code
>>>>> contribution
>> is
>>>>> still needed I would be glad to do it if someone could mentor
>>>>> me on
>> the
>>>>> task.
>>>>>
>>>>>
>>>>> Thanks,
>>>>> André Oriani
>>>>>
>>>>>
>>>>> --
>>>>> André Oriani
>>>>> MSc candidate at Computing Institute
>>>>> State University of Campinas - Brazil
>>>>> (temporarily at Sunnyvale, CA)
>>>>>
>>>>
>>>>
>>>
>>
flavio
junqueira
research scientist
[EMAIL PROTECTED]
direct +34 93-183-8828
avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300 fax (408) 349 3301
-
Re: Extracting Zab from ZookeeperAndré Oriani 2011-01-28, 07:24
Sorry for the very late reply but I am having a hard time lately. Flavio, I
am not sure if I understood your point. I know that Zookeeper and Zab are
very tied. The snapshot for Zab is the serialized version of ZooKepper's
tree and the proposal id, zxid, is intensively used by Zookeeper
application. Are you meaning that it will not be possible to get two very
distinct layers : Zab atomic broadcast protocol and ZooKeper application?
And I will not be able to use the atomic broadcast layer in isolate ?
Thanks and Regards,
André
On Sat, Jan 22, 2011 at 07:26, Flavio Junqueira <[EMAIL PROTECTED]> wrote:
> Not only it won't be trivial, but I don't have it completely clear that if
> we separate Zab and ZooKeeper you will have exactly what you need. In our
> previous discussions about this issue, we ended up concluding that we would
> need a richer interface than simply abcast and abdeliver. For example, one
> important aspect currently is stable storage for logging and checkpointing.
> To separate the ZooKeeper application and Zab we would have to decide which
> side would manage it, but both sides would need access to to it some way.
>
> It sounds like a good idea to think carefully if a separate Zab would
> really satisfy your requirements before spending time on the
> implementation.
>
> -Flavio
>
> On Jan 22, 2011, at 8:43 AM, André Oriani wrote:
>
> If I do not reply soon, do not think I gave up. My employer is keeping me
> busy lately :)
>
> I will try take a look at the patch of jira
>
> Regards,
> André
>
> On Thu, Jan 20, 2011 at 09:45, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>
> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
>
> finished work product from the original effort. I suspect that this
>
> would be a great starting point, most of the work being to "port"
>
> those changes onto the latest trunk. Ben what do you think? Was the
>
> original approach solid? Seems like a great starting point.
>
>
> Patrick
>
>
> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]> wrote:
>
> Sure, I am interested in. What should I do besides reading the papers,
>
> downloading the code compiling and doing some code walkthrough ?
>
>
>
> Tks,
>
> André
>
>
> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]>
>
> wrote:
>
>
> it's funny, i was just thinking about this yesterday.
>
>
> no one is working on it, so it is still open.
>
>
> it is a non-trivial piece of work, but i'd be willing to give guidance
>
> if
>
> you are interested in it.
>
>
> ben
>
>
> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
>
>
>
> On 01/18/2011 11:24 PM, André Oriani wrote:
>
>
> Hi,
>
>
> I need to implement a replication protocol and therefore I need total
>
> order
>
> broadcast primitive. I was thinking of using Zab. But I saw that the
>
> broadcast protocol is very intermixed with Zookeeper's tree. I heard
>
> of
>
> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution
>
> is
>
> still needed I would be glad to do it if someone could mentor me on
>
> the
>
> task.
>
>
>
> Thanks,
>
> André Oriani
>
>
>
> --
>
> André Oriani
>
> MSc candidate at Computing Institute
>
> State University of Campinas - Brazil
>
> (temporarily at Sunnyvale, CA)
>
>
>
>
>
>
>
> *flavio*
> *junqueira*
>
> research scientist
>
> [EMAIL PROTECTED]
> direct +34 93-183-8828 <tel:+34931838828>
>
> avinguda diagonal 177, 8th floor, barcelona, 08018, es
> phone (408) 349 3300 <tel:+14083493300> fax (408) 349 3301<tel:+14083493301>
>
>
>
am not sure if I understood your point. I know that Zookeeper and Zab are
very tied. The snapshot for Zab is the serialized version of ZooKepper's
tree and the proposal id, zxid, is intensively used by Zookeeper
application. Are you meaning that it will not be possible to get two very
distinct layers : Zab atomic broadcast protocol and ZooKeper application?
And I will not be able to use the atomic broadcast layer in isolate ?
Thanks and Regards,
André
On Sat, Jan 22, 2011 at 07:26, Flavio Junqueira <[EMAIL PROTECTED]> wrote:
> Not only it won't be trivial, but I don't have it completely clear that if
> we separate Zab and ZooKeeper you will have exactly what you need. In our
> previous discussions about this issue, we ended up concluding that we would
> need a richer interface than simply abcast and abdeliver. For example, one
> important aspect currently is stable storage for logging and checkpointing.
> To separate the ZooKeeper application and Zab we would have to decide which
> side would manage it, but both sides would need access to to it some way.
>
> It sounds like a good idea to think carefully if a separate Zab would
> really satisfy your requirements before spending time on the
> implementation.
>
> -Flavio
>
> On Jan 22, 2011, at 8:43 AM, André Oriani wrote:
>
> If I do not reply soon, do not think I gave up. My employer is keeping me
> busy lately :)
>
> I will try take a look at the patch of jira
>
> Regards,
> André
>
> On Thu, Jan 20, 2011 at 09:45, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>
> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
>
> finished work product from the original effort. I suspect that this
>
> would be a great starting point, most of the work being to "port"
>
> those changes onto the latest trunk. Ben what do you think? Was the
>
> original approach solid? Seems like a great starting point.
>
>
> Patrick
>
>
> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]> wrote:
>
> Sure, I am interested in. What should I do besides reading the papers,
>
> downloading the code compiling and doing some code walkthrough ?
>
>
>
> Tks,
>
> André
>
>
> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]>
>
> wrote:
>
>
> it's funny, i was just thinking about this yesterday.
>
>
> no one is working on it, so it is still open.
>
>
> it is a non-trivial piece of work, but i'd be willing to give guidance
>
> if
>
> you are interested in it.
>
>
> ben
>
>
> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
>
>
>
> On 01/18/2011 11:24 PM, André Oriani wrote:
>
>
> Hi,
>
>
> I need to implement a replication protocol and therefore I need total
>
> order
>
> broadcast primitive. I was thinking of using Zab. But I saw that the
>
> broadcast protocol is very intermixed with Zookeeper's tree. I heard
>
> of
>
> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution
>
> is
>
> still needed I would be glad to do it if someone could mentor me on
>
> the
>
> task.
>
>
>
> Thanks,
>
> André Oriani
>
>
>
> --
>
> André Oriani
>
> MSc candidate at Computing Institute
>
> State University of Campinas - Brazil
>
> (temporarily at Sunnyvale, CA)
>
>
>
>
>
>
>
> *flavio*
> *junqueira*
>
> research scientist
>
> [EMAIL PROTECTED]
> direct +34 93-183-8828 <tel:+34931838828>
>
> avinguda diagonal 177, 8th floor, barcelona, 08018, es
> phone (408) 349 3300 <tel:+14083493300> fax (408) 349 3301<tel:+14083493301>
>
>
>
-
Re: Extracting Zab from ZookeeperTed Dunning 2011-01-28, 15:03
I think that what Flavio was saying is that it is like pulling a string on a
sweater. Almost any application that wants ZAB is probably going to need
the broadcast and they the broadcast, they will want to have logging. And
transactions. And so on.
Moreover, some of these features are not necessarily encoded in ZK in a way
that you can point to. Instead they are enabled by the way that several
functional units are glued together.
On Thu, Jan 27, 2011 at 11:24 PM, André Oriani <[EMAIL PROTECTED]> wrote:
> Sorry for the very late reply but I am having a hard time lately. Flavio,
> I am not sure if I understood your point. I know that Zookeeper and Zab are
> very tied. The snapshot for Zab is the serialized version of ZooKepper's
> tree and the proposal id, zxid, is intensively used by Zookeeper
> application. Are you meaning that it will not be possible to get two very
> distinct layers : Zab atomic broadcast protocol and ZooKeper application?
> And I will not be able to use the atomic broadcast layer in isolate ?
>
>
> Thanks and Regards,
> André
>
> On Sat, Jan 22, 2011 at 07:26, Flavio Junqueira <[EMAIL PROTECTED]> wrote:
>
>> Not only it won't be trivial, but I don't have it completely clear that if
>> we separate Zab and ZooKeeper you will have exactly what you need. In our
>> previous discussions about this issue, we ended up concluding that we would
>> need a richer interface than simply abcast and abdeliver. For example, one
>> important aspect currently is stable storage for logging and checkpointing.
>> To separate the ZooKeeper application and Zab we would have to decide which
>> side would manage it, but both sides would need access to to it some way.
>>
>> It sounds like a good idea to think carefully if a separate Zab would
>> really satisfy your requirements before spending time on the
>> implementation.
>>
>> -Flavio
>>
>> On Jan 22, 2011, at 8:43 AM, André Oriani wrote:
>>
>> If I do not reply soon, do not think I gave up. My employer is keeping me
>> busy lately :)
>>
>> I will try take a look at the patch of jira
>>
>> Regards,
>> André
>>
>> On Thu, Jan 20, 2011 at 09:45, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>>
>> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
>>
>> finished work product from the original effort. I suspect that this
>>
>> would be a great starting point, most of the work being to "port"
>>
>> those changes onto the latest trunk. Ben what do you think? Was the
>>
>> original approach solid? Seems like a great starting point.
>>
>>
>> Patrick
>>
>>
>> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]> wrote:
>>
>> Sure, I am interested in. What should I do besides reading the papers,
>>
>> downloading the code compiling and doing some code walkthrough ?
>>
>>
>>
>> Tks,
>>
>> André
>>
>>
>> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]>
>>
>> wrote:
>>
>>
>> it's funny, i was just thinking about this yesterday.
>>
>>
>> no one is working on it, so it is still open.
>>
>>
>> it is a non-trivial piece of work, but i'd be willing to give guidance
>>
>> if
>>
>> you are interested in it.
>>
>>
>> ben
>>
>>
>> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
>>
>>
>>
>> On 01/18/2011 11:24 PM, André Oriani wrote:
>>
>>
>> Hi,
>>
>>
>> I need to implement a replication protocol and therefore I need total
>>
>> order
>>
>> broadcast primitive. I was thinking of using Zab. But I saw that the
>>
>> broadcast protocol is very intermixed with Zookeeper's tree. I heard
>>
>> of
>>
>> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution
>>
>> is
>>
>> still needed I would be glad to do it if someone could mentor me on
>>
>> the
>>
>> task.
>>
>>
>>
>> Thanks,
>>
>> André Oriani
>>
>>
>>
>> --
>>
>> André Oriani
>>
>> MSc candidate at Computing Institute
>>
>> State University of Campinas - Brazil
>>
>> (temporarily at Sunnyvale, CA)
>>
>>
>>
>>
>>
>>
>>
>> *flavio*
>> *junqueira*
>>
>> research scientist
sweater. Almost any application that wants ZAB is probably going to need
the broadcast and they the broadcast, they will want to have logging. And
transactions. And so on.
Moreover, some of these features are not necessarily encoded in ZK in a way
that you can point to. Instead they are enabled by the way that several
functional units are glued together.
On Thu, Jan 27, 2011 at 11:24 PM, André Oriani <[EMAIL PROTECTED]> wrote:
> Sorry for the very late reply but I am having a hard time lately. Flavio,
> I am not sure if I understood your point. I know that Zookeeper and Zab are
> very tied. The snapshot for Zab is the serialized version of ZooKepper's
> tree and the proposal id, zxid, is intensively used by Zookeeper
> application. Are you meaning that it will not be possible to get two very
> distinct layers : Zab atomic broadcast protocol and ZooKeper application?
> And I will not be able to use the atomic broadcast layer in isolate ?
>
>
> Thanks and Regards,
> André
>
> On Sat, Jan 22, 2011 at 07:26, Flavio Junqueira <[EMAIL PROTECTED]> wrote:
>
>> Not only it won't be trivial, but I don't have it completely clear that if
>> we separate Zab and ZooKeeper you will have exactly what you need. In our
>> previous discussions about this issue, we ended up concluding that we would
>> need a richer interface than simply abcast and abdeliver. For example, one
>> important aspect currently is stable storage for logging and checkpointing.
>> To separate the ZooKeeper application and Zab we would have to decide which
>> side would manage it, but both sides would need access to to it some way.
>>
>> It sounds like a good idea to think carefully if a separate Zab would
>> really satisfy your requirements before spending time on the
>> implementation.
>>
>> -Flavio
>>
>> On Jan 22, 2011, at 8:43 AM, André Oriani wrote:
>>
>> If I do not reply soon, do not think I gave up. My employer is keeping me
>> busy lately :)
>>
>> I will try take a look at the patch of jira
>>
>> Regards,
>> André
>>
>> On Thu, Jan 20, 2011 at 09:45, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>>
>> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
>>
>> finished work product from the original effort. I suspect that this
>>
>> would be a great starting point, most of the work being to "port"
>>
>> those changes onto the latest trunk. Ben what do you think? Was the
>>
>> original approach solid? Seems like a great starting point.
>>
>>
>> Patrick
>>
>>
>> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani <[EMAIL PROTECTED]> wrote:
>>
>> Sure, I am interested in. What should I do besides reading the papers,
>>
>> downloading the code compiling and doing some code walkthrough ?
>>
>>
>>
>> Tks,
>>
>> André
>>
>>
>> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed <[EMAIL PROTECTED]>
>>
>> wrote:
>>
>>
>> it's funny, i was just thinking about this yesterday.
>>
>>
>> no one is working on it, so it is still open.
>>
>>
>> it is a non-trivial piece of work, but i'd be willing to give guidance
>>
>> if
>>
>> you are interested in it.
>>
>>
>> ben
>>
>>
>> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
>>
>>
>>
>> On 01/18/2011 11:24 PM, André Oriani wrote:
>>
>>
>> Hi,
>>
>>
>> I need to implement a replication protocol and therefore I need total
>>
>> order
>>
>> broadcast primitive. I was thinking of using Zab. But I saw that the
>>
>> broadcast protocol is very intermixed with Zookeeper's tree. I heard
>>
>> of
>>
>> ZOOKEEPER-30 . What happen of it ? It still open. If code contribution
>>
>> is
>>
>> still needed I would be glad to do it if someone could mentor me on
>>
>> the
>>
>> task.
>>
>>
>>
>> Thanks,
>>
>> André Oriani
>>
>>
>>
>> --
>>
>> André Oriani
>>
>> MSc candidate at Computing Institute
>>
>> State University of Campinas - Brazil
>>
>> (temporarily at Sunnyvale, CA)
>>
>>
>>
>>
>>
>>
>>
>> *flavio*
>> *junqueira*
>>
>> research scientist
-
Re: Extracting Zab from ZookeeperBenjamin Reed 2011-01-28, 18:27
flavio and ted are correct. zab and zookeeper are tightly coupled, and
as flavio points out, zab provides a bit more than just atomic broadcast.
i think it would be a good thing to do for three reasons:
1) it would make testing, benchmarking, and fixing that layer
(especially for ZOOKEEPER-22) much easier.
2) we could try different backends easier. (i'd love to try using
bookkeeper for example.)
3) although the zab interface must be richer than normal atomic
broadcast (access to past transactions, for example), i think in
practice applications that do primary/backup can benefit from this
richer interface.
ben
On 01/28/2011 07:03 AM, Ted Dunning wrote:
> I think that what Flavio was saying is that it is like pulling a string on a
> sweater. Almost any application that wants ZAB is probably going to need
> the broadcast and they the broadcast, they will want to have logging. And
> transactions. And so on.
>
> Moreover, some of these features are not necessarily encoded in ZK in a way
> that you can point to. Instead they are enabled by the way that several
> functional units are glued together.
>
> On Thu, Jan 27, 2011 at 11:24 PM, André Oriani<[EMAIL PROTECTED]> wrote:
>
>> Sorry for the very late reply but I am having a hard time lately. Flavio,
>> I am not sure if I understood your point. I know that Zookeeper and Zab are
>> very tied. The snapshot for Zab is the serialized version of ZooKepper's
>> tree and the proposal id, zxid, is intensively used by Zookeeper
>> application. Are you meaning that it will not be possible to get two very
>> distinct layers : Zab atomic broadcast protocol and ZooKeper application?
>> And I will not be able to use the atomic broadcast layer in isolate ?
>>
>>
>> Thanks and Regards,
>> André
>>
>> On Sat, Jan 22, 2011 at 07:26, Flavio Junqueira<[EMAIL PROTECTED]> wrote:
>>
>>> Not only it won't be trivial, but I don't have it completely clear that if
>>> we separate Zab and ZooKeeper you will have exactly what you need. In our
>>> previous discussions about this issue, we ended up concluding that we would
>>> need a richer interface than simply abcast and abdeliver. For example, one
>>> important aspect currently is stable storage for logging and checkpointing.
>>> To separate the ZooKeeper application and Zab we would have to decide which
>>> side would manage it, but both sides would need access to to it some way.
>>>
>>> It sounds like a good idea to think carefully if a separate Zab would
>>> really satisfy your requirements before spending time on the
>>> implementation.
>>>
>>> -Flavio
>>>
>>> On Jan 22, 2011, at 8:43 AM, André Oriani wrote:
>>>
>>> If I do not reply soon, do not think I gave up. My employer is keeping me
>>> busy lately :)
>>>
>>> I will try take a look at the patch of jira
>>>
>>> Regards,
>>> André
>>>
>>> On Thu, Jan 20, 2011 at 09:45, Patrick Hunt<[EMAIL PROTECTED]> wrote:
>>>
>>> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
>>>
>>> finished work product from the original effort. I suspect that this
>>>
>>> would be a great starting point, most of the work being to "port"
>>>
>>> those changes onto the latest trunk. Ben what do you think? Was the
>>>
>>> original approach solid? Seems like a great starting point.
>>>
>>>
>>> Patrick
>>>
>>>
>>> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani<[EMAIL PROTECTED]> wrote:
>>>
>>> Sure, I am interested in. What should I do besides reading the papers,
>>>
>>> downloading the code compiling and doing some code walkthrough ?
>>>
>>>
>>>
>>> Tks,
>>>
>>> André
>>>
>>>
>>> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed<[EMAIL PROTECTED]>
>>>
>>> wrote:
>>>
>>>
>>> it's funny, i was just thinking about this yesterday.
>>>
>>>
>>> no one is working on it, so it is still open.
>>>
>>>
>>> it is a non-trivial piece of work, but i'd be willing to give guidance
>>>
>>> if
>>>
>>> you are interested in it.
>>>
>>>
>>> ben
>>>
>>>
>>> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
as flavio points out, zab provides a bit more than just atomic broadcast.
i think it would be a good thing to do for three reasons:
1) it would make testing, benchmarking, and fixing that layer
(especially for ZOOKEEPER-22) much easier.
2) we could try different backends easier. (i'd love to try using
bookkeeper for example.)
3) although the zab interface must be richer than normal atomic
broadcast (access to past transactions, for example), i think in
practice applications that do primary/backup can benefit from this
richer interface.
ben
On 01/28/2011 07:03 AM, Ted Dunning wrote:
> I think that what Flavio was saying is that it is like pulling a string on a
> sweater. Almost any application that wants ZAB is probably going to need
> the broadcast and they the broadcast, they will want to have logging. And
> transactions. And so on.
>
> Moreover, some of these features are not necessarily encoded in ZK in a way
> that you can point to. Instead they are enabled by the way that several
> functional units are glued together.
>
> On Thu, Jan 27, 2011 at 11:24 PM, André Oriani<[EMAIL PROTECTED]> wrote:
>
>> Sorry for the very late reply but I am having a hard time lately. Flavio,
>> I am not sure if I understood your point. I know that Zookeeper and Zab are
>> very tied. The snapshot for Zab is the serialized version of ZooKepper's
>> tree and the proposal id, zxid, is intensively used by Zookeeper
>> application. Are you meaning that it will not be possible to get two very
>> distinct layers : Zab atomic broadcast protocol and ZooKeper application?
>> And I will not be able to use the atomic broadcast layer in isolate ?
>>
>>
>> Thanks and Regards,
>> André
>>
>> On Sat, Jan 22, 2011 at 07:26, Flavio Junqueira<[EMAIL PROTECTED]> wrote:
>>
>>> Not only it won't be trivial, but I don't have it completely clear that if
>>> we separate Zab and ZooKeeper you will have exactly what you need. In our
>>> previous discussions about this issue, we ended up concluding that we would
>>> need a richer interface than simply abcast and abdeliver. For example, one
>>> important aspect currently is stable storage for logging and checkpointing.
>>> To separate the ZooKeeper application and Zab we would have to decide which
>>> side would manage it, but both sides would need access to to it some way.
>>>
>>> It sounds like a good idea to think carefully if a separate Zab would
>>> really satisfy your requirements before spending time on the
>>> implementation.
>>>
>>> -Flavio
>>>
>>> On Jan 22, 2011, at 8:43 AM, André Oriani wrote:
>>>
>>> If I do not reply soon, do not think I gave up. My employer is keeping me
>>> busy lately :)
>>>
>>> I will try take a look at the patch of jira
>>>
>>> Regards,
>>> André
>>>
>>> On Thu, Jan 20, 2011 at 09:45, Patrick Hunt<[EMAIL PROTECTED]> wrote:
>>>
>>> It's non-trivial - but iirc the patches/links on ZOOKEEPER-30 show the
>>>
>>> finished work product from the original effort. I suspect that this
>>>
>>> would be a great starting point, most of the work being to "port"
>>>
>>> those changes onto the latest trunk. Ben what do you think? Was the
>>>
>>> original approach solid? Seems like a great starting point.
>>>
>>>
>>> Patrick
>>>
>>>
>>> On Thu, Jan 20, 2011 at 8:35 AM, André Oriani<[EMAIL PROTECTED]> wrote:
>>>
>>> Sure, I am interested in. What should I do besides reading the papers,
>>>
>>> downloading the code compiling and doing some code walkthrough ?
>>>
>>>
>>>
>>> Tks,
>>>
>>> André
>>>
>>>
>>> On Wed, Jan 19, 2011 at 11:32, Benjamin Reed<[EMAIL PROTECTED]>
>>>
>>> wrote:
>>>
>>>
>>> it's funny, i was just thinking about this yesterday.
>>>
>>>
>>> no one is working on it, so it is still open.
>>>
>>>
>>> it is a non-trivial piece of work, but i'd be willing to give guidance
>>>
>>> if
>>>
>>> you are interested in it.
>>>
>>>
>>> ben
>>>
>>>
>>> ps - btw, this is definitely a [EMAIL PROTECTED] discussion :)
-
Re: Extracting Zab from ZookeeperAndrew Purtell 2011-01-31, 00:56
FWIW, we have this crazy idea of using something like an extracted ZAB for maintaining quorum-cliques in a modified HBase cluster: https://issues.apache.org/jira/browse/HBASE-2357
- Andy
> From: Benjamin Reed <[EMAIL PROTECTED]>
> Subject: Re: Extracting Zab from Zookeeper
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Cc: "Ted Dunning" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Date: Friday, January 28, 2011, 10:27 AM
> flavio and ted are correct. zab and
> zookeeper are tightly coupled, and
> as flavio points out, zab provides a bit more than just
> atomic broadcast.
>
> i think it would be a good thing to do for three reasons:
>
> 1) it would make testing, benchmarking, and fixing that layer
> (especially for ZOOKEEPER-22) much easier.
> 2) we could try different backends easier. (i'd love to try
> using bookkeeper for example.)
> 3) although the zab interface must be richer than normal
> atomic broadcast (access to past transactions, for example), i
> think in practice applications that do primary/backup can benefit
> from this richer interface.
>
> ben
>
> On 01/28/2011 07:03 AM, Ted Dunning wrote:
> > I think that what Flavio was saying is that it is like
> > pulling a string on a sweater. Almost any application that
> > wants ZAB is probably going to need the broadcast and they the
> > broadcast, they will want to have logging. And transactions.
> > And so on.
> >
> > Moreover, some of these features are not necessarily
> > encoded in ZK in a way that you can point to. Instead they are
> > enabled by the way that several functional units are glued
> > together.
- Andy
> From: Benjamin Reed <[EMAIL PROTECTED]>
> Subject: Re: Extracting Zab from Zookeeper
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Cc: "Ted Dunning" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Date: Friday, January 28, 2011, 10:27 AM
> flavio and ted are correct. zab and
> zookeeper are tightly coupled, and
> as flavio points out, zab provides a bit more than just
> atomic broadcast.
>
> i think it would be a good thing to do for three reasons:
>
> 1) it would make testing, benchmarking, and fixing that layer
> (especially for ZOOKEEPER-22) much easier.
> 2) we could try different backends easier. (i'd love to try
> using bookkeeper for example.)
> 3) although the zab interface must be richer than normal
> atomic broadcast (access to past transactions, for example), i
> think in practice applications that do primary/backup can benefit
> from this richer interface.
>
> ben
>
> On 01/28/2011 07:03 AM, Ted Dunning wrote:
> > I think that what Flavio was saying is that it is like
> > pulling a string on a sweater. Almost any application that
> > wants ZAB is probably going to need the broadcast and they the
> > broadcast, they will want to have logging. And transactions.
> > And so on.
> >
> > Moreover, some of these features are not necessarily
> > encoded in ZK in a way that you can point to. Instead they are
> > enabled by the way that several functional units are glued
> > together.
-
Re: Extracting Zab from ZookeeperMahadev Konar 2011-01-31, 03:21
Removing user@apache,
Wow... That¹s pretty interesting. Unfortunately the apache site is down. But
reading from the cached archives, it feels like the jira is about
master-slave for read only replica of the region servers?
Thanks
Mahadev
On 1/30/11 4:56 PM, "Andrew Purtell" <[EMAIL PROTECTED]> wrote:
> FWIW, we have this crazy idea of using something like an extracted ZAB for
> maintaining quorum-cliques in a modified HBase cluster:
> https://issues.apache.org/jira/browse/HBASE-2357
>
> - Andy
>
>> From: Benjamin Reed <[EMAIL PROTECTED]>
>> Subject: Re: Extracting Zab from Zookeeper
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Cc: "Ted Dunning" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]"
>> <[EMAIL PROTECTED]>
>> Date: Friday, January 28, 2011, 10:27 AM
>> flavio and ted are correct. zab and
>> zookeeper are tightly coupled, and
>> as flavio points out, zab provides a bit more than just
>> atomic broadcast.
>>
>> i think it would be a good thing to do for three reasons:
>>
>> 1) it would make testing, benchmarking, and fixing that layer
>> (especially for ZOOKEEPER-22) much easier.
>> 2) we could try different backends easier. (i'd love to try
>> using bookkeeper for example.)
>> 3) although the zab interface must be richer than normal
>> atomic broadcast (access to past transactions, for example), i
>> think in practice applications that do primary/backup can benefit
>> from this richer interface.
>>
>> ben
>>
>> On 01/28/2011 07:03 AM, Ted Dunning wrote:
>>> I think that what Flavio was saying is that it is like
>>> pulling a string on a sweater. Almost any application that
>>> wants ZAB is probably going to need the broadcast and they the
>>> broadcast, they will want to have logging. And transactions.
>>> And so on.
>>>
>>> Moreover, some of these features are not necessarily
>>> encoded in ZK in a way that you can point to. Instead they are
>>> enabled by the way that several functional units are glued
>>> together.
>
>
>
>
>
Wow... That¹s pretty interesting. Unfortunately the apache site is down. But
reading from the cached archives, it feels like the jira is about
master-slave for read only replica of the region servers?
Thanks
Mahadev
On 1/30/11 4:56 PM, "Andrew Purtell" <[EMAIL PROTECTED]> wrote:
> FWIW, we have this crazy idea of using something like an extracted ZAB for
> maintaining quorum-cliques in a modified HBase cluster:
> https://issues.apache.org/jira/browse/HBASE-2357
>
> - Andy
>
>> From: Benjamin Reed <[EMAIL PROTECTED]>
>> Subject: Re: Extracting Zab from Zookeeper
>> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
>> Cc: "Ted Dunning" <[EMAIL PROTECTED]>, "[EMAIL PROTECTED]"
>> <[EMAIL PROTECTED]>
>> Date: Friday, January 28, 2011, 10:27 AM
>> flavio and ted are correct. zab and
>> zookeeper are tightly coupled, and
>> as flavio points out, zab provides a bit more than just
>> atomic broadcast.
>>
>> i think it would be a good thing to do for three reasons:
>>
>> 1) it would make testing, benchmarking, and fixing that layer
>> (especially for ZOOKEEPER-22) much easier.
>> 2) we could try different backends easier. (i'd love to try
>> using bookkeeper for example.)
>> 3) although the zab interface must be richer than normal
>> atomic broadcast (access to past transactions, for example), i
>> think in practice applications that do primary/backup can benefit
>> from this richer interface.
>>
>> ben
>>
>> On 01/28/2011 07:03 AM, Ted Dunning wrote:
>>> I think that what Flavio was saying is that it is like
>>> pulling a string on a sweater. Almost any application that
>>> wants ZAB is probably going to need the broadcast and they the
>>> broadcast, they will want to have logging. And transactions.
>>> And so on.
>>>
>>> Moreover, some of these features are not necessarily
>>> encoded in ZK in a way that you can point to. Instead they are
>>> enabled by the way that several functional units are glued
>>> together.
>
>
>
>
>
-
Re: Extracting Zab from ZookeeperAndrew Purtell 2011-02-01, 16:38
> From: Mahadev Konar <[EMAIL PROTECTED]>
>
> Wow... That¹s pretty interesting. Unfortunately the apache site
> is down. But reading from the cached archives, it feels like the
> jira is about master-slave for read only replica of the region
> servers?
Two ideas actually:
1) Do pretty straightforward log shipping from region master to read only replicas.
2) Divide the cluster into quorum 3-cliques. Extract ZAB and use it to maintain consensus on writes from region master to two read only replicas. Run the consensus protocol in parallel with HDFS hflush to the write ahead log. Needs a lot of work filling in the detail, obviously, but that's the general notion.
#1 is relatively simple but trades away the consistency for which HBase is indicated for higher availability (for reads) when regions are in transition.
#2 is not simple at all but may let maintain replicas that are fully consistent at all times with the region master, not lower region master write performance unacceptably, and also gain the higher availability (for reads) when regions are in transition.
Both #1 and #2 would be offered as options to the system implementer.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back.
- Piet Hein (via Tom White)
>
> Wow... That¹s pretty interesting. Unfortunately the apache site
> is down. But reading from the cached archives, it feels like the
> jira is about master-slave for read only replica of the region
> servers?
Two ideas actually:
1) Do pretty straightforward log shipping from region master to read only replicas.
2) Divide the cluster into quorum 3-cliques. Extract ZAB and use it to maintain consensus on writes from region master to two read only replicas. Run the consensus protocol in parallel with HDFS hflush to the write ahead log. Needs a lot of work filling in the detail, obviously, but that's the general notion.
#1 is relatively simple but trades away the consistency for which HBase is indicated for higher availability (for reads) when regions are in transition.
#2 is not simple at all but may let maintain replicas that are fully consistent at all times with the region master, not lower region master write performance unacceptably, and also gain the higher availability (for reads) when regions are in transition.
Both #1 and #2 would be offered as options to the system implementer.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back.
- Piet Hein (via Tom White)
-
Re: Extracting Zab from ZookeeperFlavio Junqueira 2011-02-02, 10:14
Hi Andrew, Interesting use case, thanks for sharing. I'm curious
about a few things:
On Feb 1, 2011, at 5:38 PM, Andrew Purtell wrote:
> Two ideas actually:
>
> 1) Do pretty straightforward log shipping from region master to read
> only replicas.
>
I don't understand why you have to ship the log to the read only
replicas. Aren't you storing the log on HDFS currently? Can't they
read from HDFS directly?
> 2) Divide the cluster into quorum 3-cliques. Extract ZAB and use it
> to maintain consensus on writes from region master to two read only
> replicas. Run the consensus protocol in parallel with HDFS hflush to
> the write ahead log. Needs a lot of work filling in the detail,
> obviously, but that's the general notion.
>
I wonder why you are choosing 3 for the size of a clique and not
letting it be a free parameter. I would think that this a decision of
the user. Are you choosing 3 to avoid the replication overhead?
> #1 is relatively simple but trades away the consistency for which
> HBase is indicated for higher availability (for reads) when regions
> are in transition.
>
I don't see where you could have inconsistencies here. Would you mind
elaborating a bit further?
> #2 is not simple at all but may let maintain replicas that are fully
> consistent at all times with the region master, not lower region
> master write performance unacceptably, and also gain the higher
> availability (for reads) when regions are in transition.
>
Agreed, it will be tricky, especially because we would have to extract
Zab first.
Cheers,
-Flavio
>
> Problems worthy of attack prove their worth by hitting back.
> - Piet Hein (via Tom White)
>
>
>
>
>
flavio
junqueira
research scientist
[EMAIL PROTECTED]
direct +34 93-183-8828
avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300 fax (408) 349 3301
about a few things:
On Feb 1, 2011, at 5:38 PM, Andrew Purtell wrote:
> Two ideas actually:
>
> 1) Do pretty straightforward log shipping from region master to read
> only replicas.
>
I don't understand why you have to ship the log to the read only
replicas. Aren't you storing the log on HDFS currently? Can't they
read from HDFS directly?
> 2) Divide the cluster into quorum 3-cliques. Extract ZAB and use it
> to maintain consensus on writes from region master to two read only
> replicas. Run the consensus protocol in parallel with HDFS hflush to
> the write ahead log. Needs a lot of work filling in the detail,
> obviously, but that's the general notion.
>
I wonder why you are choosing 3 for the size of a clique and not
letting it be a free parameter. I would think that this a decision of
the user. Are you choosing 3 to avoid the replication overhead?
> #1 is relatively simple but trades away the consistency for which
> HBase is indicated for higher availability (for reads) when regions
> are in transition.
>
I don't see where you could have inconsistencies here. Would you mind
elaborating a bit further?
> #2 is not simple at all but may let maintain replicas that are fully
> consistent at all times with the region master, not lower region
> master write performance unacceptably, and also gain the higher
> availability (for reads) when regions are in transition.
>
Agreed, it will be tricky, especially because we would have to extract
Zab first.
Cheers,
-Flavio
>
> Problems worthy of attack prove their worth by hitting back.
> - Piet Hein (via Tom White)
>
>
>
>
>
flavio
junqueira
research scientist
[EMAIL PROTECTED]
direct +34 93-183-8828
avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300 fax (408) 349 3301
-
Re: Extracting Zab from ZookeeperAndrew Purtell 2011-02-03, 18:52
Hi Flavio,
> I don't understand why you have to ship the log to the read only replicas. Aren't you storing the log on HDFS currently? Can't they read from HDFS directly?
Possibly the replicas can "tail" the WAL of the master, was using the term log shipping in the abstract. However I'm not an HDFS expert so unsure if we could read the last (partial) block in the WAL. Newly written data exists only in memory so the WAL would be the only option for transmitting this data until flush without some sort of direct replication.
> I wonder why you are choosing 3 for the size of a clique and not letting it be a free parameter.
It would but 3 seems a reasonable default. (?)
> Are you choosing 3 to avoid the replication overhead?
Yes.
> #1 is relatively simple but trades away the consistency
> I don't see where you could have inconsistencies here. Would you mind elaborating a bit further?
At any given instant queries to a replica may not return the same result as the (write) master for data in memstore and (possibly) in the last block of the WAL.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back.
- Piet Hein (via Tom White)
--- On Wed, 2/2/11, Flavio Junqueira <[EMAIL PROTECTED]> wrote:
From: Flavio Junqueira <[EMAIL PROTECTED]>
Subject: Re: Extracting Zab from Zookeeper
Date: Wednesday, February 2, 2011, 2:14 AM
Hi Andrew, Interesting use case, thanks for sharing. I'm curious about a few things:
On Feb 1, 2011, at 5:38 PM, Andrew Purtell wrote:
Two ideas actually:
1) Do pretty straightforward log shipping from region master to read only replicas.
I don't understand why you have to ship the log to the read only replicas. Aren't you storing the log on HDFS currently? Can't they read from HDFS directly?
2) Divide the cluster into quorum 3-cliques. Extract ZAB and use it to maintain consensus on writes from region master to two read only replicas. Run the consensus protocol in parallel with HDFS hflush to the write ahead log. Needs a lot of work filling in the detail, obviously, but that's the general notion.
I wonder why you are choosing 3 for the size of a clique and not letting it be a free parameter. I would think that this a decision of the user. Are you choosing 3 to avoid the replication overhead?
#1 is relatively simple but trades away the consistency for which HBase is indicated for higher availability (for reads) when regions are in transition.
I don't see where you could have inconsistencies here. Would you mind elaborating a bit further?
#2 is not simple at all but may let maintain replicas that are fully consistent at all times with the region master, not lower region master write performance unacceptably, and also gain the higher availability (for reads) when regions are in transition.
Agreed, it will be tricky, especially because we would have to extract Zab first.
Cheers,-Flavio
flavio junqueira
research scientist
[EMAIL PROTECTED]
direct +34 93-183-8828
avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300 fax (408) 349 3301
> I don't understand why you have to ship the log to the read only replicas. Aren't you storing the log on HDFS currently? Can't they read from HDFS directly?
Possibly the replicas can "tail" the WAL of the master, was using the term log shipping in the abstract. However I'm not an HDFS expert so unsure if we could read the last (partial) block in the WAL. Newly written data exists only in memory so the WAL would be the only option for transmitting this data until flush without some sort of direct replication.
> I wonder why you are choosing 3 for the size of a clique and not letting it be a free parameter.
It would but 3 seems a reasonable default. (?)
> Are you choosing 3 to avoid the replication overhead?
Yes.
> #1 is relatively simple but trades away the consistency
> I don't see where you could have inconsistencies here. Would you mind elaborating a bit further?
At any given instant queries to a replica may not return the same result as the (write) master for data in memstore and (possibly) in the last block of the WAL.
Best regards,
- Andy
Problems worthy of attack prove their worth by hitting back.
- Piet Hein (via Tom White)
--- On Wed, 2/2/11, Flavio Junqueira <[EMAIL PROTECTED]> wrote:
From: Flavio Junqueira <[EMAIL PROTECTED]>
Subject: Re: Extracting Zab from Zookeeper
Date: Wednesday, February 2, 2011, 2:14 AM
Hi Andrew, Interesting use case, thanks for sharing. I'm curious about a few things:
On Feb 1, 2011, at 5:38 PM, Andrew Purtell wrote:
Two ideas actually:
1) Do pretty straightforward log shipping from region master to read only replicas.
I don't understand why you have to ship the log to the read only replicas. Aren't you storing the log on HDFS currently? Can't they read from HDFS directly?
2) Divide the cluster into quorum 3-cliques. Extract ZAB and use it to maintain consensus on writes from region master to two read only replicas. Run the consensus protocol in parallel with HDFS hflush to the write ahead log. Needs a lot of work filling in the detail, obviously, but that's the general notion.
I wonder why you are choosing 3 for the size of a clique and not letting it be a free parameter. I would think that this a decision of the user. Are you choosing 3 to avoid the replication overhead?
#1 is relatively simple but trades away the consistency for which HBase is indicated for higher availability (for reads) when regions are in transition.
I don't see where you could have inconsistencies here. Would you mind elaborating a bit further?
#2 is not simple at all but may let maintain replicas that are fully consistent at all times with the region master, not lower region master write performance unacceptably, and also gain the higher availability (for reads) when regions are in transition.
Agreed, it will be tricky, especially because we would have to extract Zab first.
Cheers,-Flavio
flavio junqueira
research scientist
[EMAIL PROTECTED]
direct +34 93-183-8828
avinguda diagonal 177, 8th floor, barcelona, 08018, es
phone (408) 349 3300 fax (408) 349 3301
-
Extracting Zab from ZookeeperAndré Oriani 2011-06-11, 02:36
Since I was cited on the slide deck below, I think I owe the community some
status.
- My MSc research had a turnaround, so I will not need Zab for now . ( But I
will still need Zookeeper)
This means that the implementation is on hold for while.
- I change the implementation strategy. Instead of deconstructing Zookeeper
like the Harvey Mudd Clinic team did , I decided to code almost from
scratch, reusing some code and design from ZooKeeper e MultiZab. The
rational behind this decision was to have smaller and condensed code that I
could test. So I stopped working on https://github.com/aoriani/zab ( I
renamed it to https://github.com/aoriani/zookeeper-zab)
- I did no publish the code yet . I was expecting to do so when I finish the
code. Right now I have atomic broadcast working, but no logging and
therefore no recover. Anyway, if someone is interested in the code and want
to collaborate, let me know.
Regards,
André Oriani
---------- Forwarded message ----------
From: Gustavo Niemeyer <[EMAIL PROTECTED]>
Date: Fri, Jun 10, 2011 at 15:41
Subject: Debian packager orphaning ZooKeeper
To: [EMAIL PROTECTED]
Some valid points, and a high ratio of ranting.
"""
please be aware that I'm planning on orphaning ZooKeeper in the next days.
The
reasoning behind that will shortly be available as a video or slides[1] of
my
talk at BerlinBuzzWords. - In short: The code quality is of such a kind that
I
don't consider it good enough for Debian.
"""
http://lists.debian.org/debian-devel/2011/06/msg00316.html
http://berlinbuzzwords.de/sites/berlinbuzzwords.de/files/thomas_koch_zookeeper.pdf
--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter
status.
- My MSc research had a turnaround, so I will not need Zab for now . ( But I
will still need Zookeeper)
This means that the implementation is on hold for while.
- I change the implementation strategy. Instead of deconstructing Zookeeper
like the Harvey Mudd Clinic team did , I decided to code almost from
scratch, reusing some code and design from ZooKeeper e MultiZab. The
rational behind this decision was to have smaller and condensed code that I
could test. So I stopped working on https://github.com/aoriani/zab ( I
renamed it to https://github.com/aoriani/zookeeper-zab)
- I did no publish the code yet . I was expecting to do so when I finish the
code. Right now I have atomic broadcast working, but no logging and
therefore no recover. Anyway, if someone is interested in the code and want
to collaborate, let me know.
Regards,
André Oriani
---------- Forwarded message ----------
From: Gustavo Niemeyer <[EMAIL PROTECTED]>
Date: Fri, Jun 10, 2011 at 15:41
Subject: Debian packager orphaning ZooKeeper
To: [EMAIL PROTECTED]
Some valid points, and a high ratio of ranting.
"""
please be aware that I'm planning on orphaning ZooKeeper in the next days.
The
reasoning behind that will shortly be available as a video or slides[1] of
my
talk at BerlinBuzzWords. - In short: The code quality is of such a kind that
I
don't consider it good enough for Debian.
"""
http://lists.debian.org/debian-devel/2011/06/msg00316.html
http://berlinbuzzwords.de/sites/berlinbuzzwords.de/files/thomas_koch_zookeeper.pdf
--
Gustavo Niemeyer
http://niemeyer.net
http://niemeyer.net/blog
http://niemeyer.net/twitter
