Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Zookeeper, mail # dev - [DISCUSS] move hedwig/bookkeeper to subproject


Copy link to this message
-
Re: [DISCUSS] move hedwig/bookkeeper to subproject
Benjamin Reed 2011-03-16, 16:42
i think we need to move both of them out of contrib. they are just
languishing there. i agree that they could be separate projects, but
they are closer than may appear at a high level. the application that
hedwig was designed for is using it as a write ahead log. like
TeaKeeper shows often there is a requirement to both log and broadcast
changes. In some sense we have this continuum: BookKeeper (single
writer WAL), TeaKeeper (single writer WAL with broadcast), Hedwig
(multiple writer WAL with broadcast). Hedwig is also tightly
integrated with BookKeeper. The development community is also tightly
integrated. i think there are valid reasons for doing an incubator
project or for doing separate subprojects, but i think the best option
for now would be to do a subproject under zookeeper (called either
hedwig or bookkeeper) that would host both code bases.

ben

On Wed, Mar 16, 2011 at 3:14 AM, Ivan Kelly <[EMAIL PROTECTED]> wrote:
> I agree about the separation of bookkeeper and hedwig. They solve very different problems, so lumping them together feels clunky. Perhaps bookkeeper could be moved out of zookeeper first, leaving hedwig in until there's more community interest in it.
>
> -Ivan
>
> On 15 Mar 2011, at 23:58, Dhruba Borthakur wrote:
>
>> I am interested in contributing to the bookkeeper code. It would be nice to
>> have a community around it. An incubator proposal sounds good, but the
>> zk-subproject should also work well. It woud be nice to separate out hedwig
>> and bookkeeper since they have quite different functionality.
>>
>> -dhruba
>>
>> On Mon, Mar 14, 2011 at 8:56 PM, Mahadev Konar <[EMAIL PROTECTED]> wrote:
>>
>>> I like the idea of BookKeeper/Hedwig being subprojects.
>>>
>>> thanks
>>> mahadev
>>>
>>> On Thu, Mar 10, 2011 at 1:34 AM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
>>>
>>>>
>>>>
>>>> On Tue, Mar 8, 2011 at 1:01 PM, Flavio Junqueira <[EMAIL PROTECTED]
>>>> wrote:
>>>>
>>>>> I'm sorry for not replying before. I didn't feel that the message was
>>> for
>>>>> me, since it should be pretty obvious that I'm interested in those
>>>>> projects. Here are some thoughts, though:
>>>>>
>>>>> - It would be really nice to have committers for bookkeeper/hedwig;
>>>>> - It would be really nice to have independent releases for
>>>>> bookkeeper/hedwig;
>>>>> - It sounds like bookkeeper and hedwig don't always go together, and
>>> hdfs
>>>>> is an instance in which it happens. But, hedwig builds on top of
>>> bookkeeper
>>>>> (and other components), so using hedwig implies using bookkeeper.
>>>>> Consequently, if we choose only one to be a main project, perhaps
>>> bookkeeper
>>>>> would be a better choice;
>>>>>
>>>>
>>>> Perhaps one could argue that bk/hedwig fall under "distributed system
>>>> coordination" and therefore should be part of ZK? Or is that too much of
>>> a
>>>> stretch? ;-)
>>>>
>>>> RESOLVED, that the Apache ZooKeeper Project be and hereby is responsible
>>>> for the creation and maintenance of software related to distributed
>>> system
>>>> coordination; and be it further
>>>>
>>>>
>>>>> - I don't think we have anyone who could be a project lead for these
>>>>> projects right now, so it could be a problem to split up at this point.
>>> For
>>>>> this reason, a zookeeper subproject sounds like a better option compared
>>> to
>>>>> incubator, unless we are able to find a project lead.
>>>>>
>>>>> -Flavio
>>>>>
>>>>> On Mar 2, 2011, at 6:55 PM, Benjamin Reed wrote:
>>>>>
>>>>> i wanted to start a discussion about making hedwig and bookkeeper a
>>>>> subproject. (actually pat started the discussion last month in general
>>>>> about all of the contrib projects.) there are three questions, in my
>>>>> mind, that we need to answer to move forward:
>>>>>
>>>>> 1) should it be a hedwig/bookkeeper subproject, or should there be two
>>>>> separate projects? we need to build a developer community and i'm
>>>>> wondering if we should try to build a single dev community or two. the
>>>>> relationship is a bit asymmetrical: hedwig depends on bookkeeper, but