Dear All:
  The NameNode is the Single Point of hadoop,
I want to know how to HA the NameNode.

imcaptor

imcaptor wrote:
> Dear All:
>   The NameNode is the Single Point of hadoop,

It's *one* SPOF - unless your datacentre has multiple gigabit backbones
on separate switches and more than electricity supply from different
power supply grids, you have others out there. Then there's the issue of
which fault lines your facility is close to, whether it is built on the
mud flows of previous eruptions (Enumclaw, WA spring to mind), etc.

> I want to know how to HA the NameNode.

This is a major engineering project that has been discussed on the
lists. If you want to get involved in it, and have the skills -and the
testing facilities- you would be welcome. Testing HA/failover is tricky
because of the many ways things can fail.

Otherwise, focus on making sure your namenode data is streamed off onto
different disks, you have a secondary NN, and you  have a good recovery
process that includes the failure of the NN hardware

Check out the Cloudera blog (http://www.cloudera.com).  They posted
instructions for one approach.
On 7/30/2009 8:27 AM, Steve Loughran wrote:
> imcaptor wrote:
>> Dear All:
>>   The NameNode is the Single Point of hadoop,
>
> It's *one* SPOF - unless your datacentre has multiple gigabit
> backbones on separate switches and more than electricity supply from
> different power supply grids, you have others out there. Then there's
> the issue of which fault lines your facility is close to, whether it
> is built on the mud flows of previous eruptions (Enumclaw, WA spring
> to mind), etc.
>
>> I want to know how to HA the NameNode.
>
> This is a major engineering project that has been discussed on the
> lists. If you want to get involved in it, and have the skills -and the
> testing facilities- you would be welcome. Testing HA/failover is
> tricky because of the many ways things can fail.
>
> Otherwise, focus on making sure your namenode data is streamed off
> onto different disks, you have a secondary NN, and you  have a good
> recovery process that includes the failure of the NN hardware
>
>
>
>

As David mentions, you can find more information on one approach to HA for
the NN at http://www.cloudera.com/blog/2009/07/22/hadoop-ha-configuration/.

On Thu, Jul 30, 2009 at 5:39 AM, David B. Ritch <[EMAIL PROTECTED]>wrote:

> Check out the Cloudera blog (http://www.cloudera.com).  They posted
> instructions for one approach.
>
>
> On 7/30/2009 8:27 AM, Steve Loughran wrote:
> > imcaptor wrote:
> >> Dear All:
> >>   The NameNode is the Single Point of hadoop,
> >
> > It's *one* SPOF - unless your datacentre has multiple gigabit
> > backbones on separate switches and more than electricity supply from
> > different power supply grids, you have others out there. Then there's
> > the issue of which fault lines your facility is close to, whether it
> > is built on the mud flows of previous eruptions (Enumclaw, WA spring
> > to mind), etc.
> >
> >> I want to know how to HA the NameNode.
> >
> > This is a major engineering project that has been discussed on the
> > lists. If you want to get involved in it, and have the skills -and the
> > testing facilities- you would be welcome. Testing HA/failover is
> > tricky because of the many ways things can fail.
> >
> > Otherwise, focus on making sure your namenode data is streamed off
> > onto different disks, you have a secondary NN, and you  have a good
> > recovery process that includes the failure of the NN hardware
> >
> >
> >
> >
>
>