Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Zeroconf for hadoop


Copy link to this message
-
Re: Zeroconf for hadoop
Sanjay Radia 2009-02-15, 00:04


----- Original Message -----
From: Ted Dunning <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Sent: Wed Jan 28 11:51:54 2009
Subject: Re: Zeroconf for hadoop

A big positive vote for Zookeeper.

The most salient aspect of my experience using zookeeper is that
coordination and heartbeats and discovery and failure notifications all
become nearly trivial.  The most amazing thing about ZK isn't the code that
you write, it is all the code that you never have write.

On Wed, Jan 28, 2009 at 11:01 AM, Patrick Hunt <[EMAIL PROTECTED]> wrote:

> Owen O'Malley wrote:
>
>> On Jan 25, 2009, at 2:02 PM, nitesh bhatia wrote:
>>
>>  Apple provides opensource discovery service called Bonjour (zeroconf).
>>>
>>
>> I don't know enough about Zeroconf to be able to answer definitively, but
>> I suspect the hardest bit would be figuring out the approach. Of course
>> Hadoop has to continue to work on other platforms, so cross-platform
>> strategies are better.
>>
>
> Take a look at ZooKeeper (a sub-project of Hadoop):
> http://hadoop.apache.org/zookeeper/
>
> Among other features ZooKeeper provides group membership and dynamic
> configuration support -- you could modify the various Hadoop processes to
> query & register with ZooKeeper when they come up. This could be used for
> node/service discovery as well as auto configuration of the processes.
>
> Patrick
>

--
Ted Dunning, CTO
DeepDyve
4600 Bohannon Drive, Suite 220
Menlo Park, CA 94025
www.deepdyve.com
650-324-0110, ext. 738
858-414-0013 (m)