nitesh bhatia 2009-01-25, 22:02
Owen OMalley 2009-01-26, 17:46
Doug Cutting 2009-01-26, 18:35
Steve Loughran 2009-01-27, 10:52
Patrick Hunt 2009-01-28, 19:01
Ted Dunning 2009-01-28, 19:51
----- Original Message -----
From: Ted Dunning <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED] <[EMAIL PROTECTED]>
Sent: Wed Jan 28 11:51:54 2009
Subject: Re: Zeroconf for hadoop
A big positive vote for Zookeeper.
The most salient aspect of my experience using zookeeper is that
coordination and heartbeats and discovery and failure notifications all
become nearly trivial. The most amazing thing about ZK isn't the code that
you write, it is all the code that you never have write.
On Wed, Jan 28, 2009 at 11:01 AM, Patrick Hunt <[EMAIL PROTECTED]> wrote:
> Owen O'Malley wrote:
>> On Jan 25, 2009, at 2:02 PM, nitesh bhatia wrote:
>> Apple provides opensource discovery service called Bonjour (zeroconf).
>> I don't know enough about Zeroconf to be able to answer definitively, but
>> I suspect the hardest bit would be figuring out the approach. Of course
>> Hadoop has to continue to work on other platforms, so cross-platform
>> strategies are better.
> Take a look at ZooKeeper (a sub-project of Hadoop):
> Among other features ZooKeeper provides group membership and dynamic
> configuration support -- you could modify the various Hadoop processes to
> query & register with ZooKeeper when they come up. This could be used for
> node/service discovery as well as auto configuration of the processes.
Ted Dunning, CTO
4600 Bohannon Drive, Suite 220
Menlo Park, CA 94025
650-324-0110, ext. 738