Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: hadoop namenode recovery


Copy link to this message
-
Re: hadoop namenode recovery
Hi Panshul

SecondaryNameNode is rather known as check point node. At periodic intervals it merges the editlog from NN with FS image to prevent the edit log from growing too large. This is its main functionality.

At any point the SNN would have the latest fs image but not the updated edit log. If NN goes down and if you don't have an updated copy of edit log you can use the fsImage from SNN for restoring. In that case you lose the transactions in edit log.

SNN is not a backup NN it is just a check point node.
 
Two or more NN are not possible in 1.x releases but federation makes it possible with 2.x releases. Federation is for different purpose, you should be looking at hadoop HA currently with 2.x releases.
Regards
Bejoy KS

Sent from remote device, Please excuse typos

-----Original Message-----
From: Panshul Whisper <[EMAIL PROTECTED]>
Date: Mon, 14 Jan 2013 19:04:24
To: <[EMAIL PROTECTED]>; <[EMAIL PROTECTED]>
Subject: Re: hadoop namenode recovery

thank you for the reply.

Is there a way with which I can configure my cluster to switch to the
Secondary Name Node automatically in case of the Primary Name Node failure?
When I run my current Hadoop, I see the primary and secondary both Name
nodes running. I was wondering what is that Secondary Name Node for? and
where is it configured?
I was also wondering, is it possible to have two or more Name nodes running
in the same cluster?

Thanks,
Regards,
Panshul.
On Mon, Jan 14, 2013 at 6:50 PM, <[EMAIL PROTECTED]> wrote:

> **
> Hi Panshul,
>
> Usually for reliability there will be multiple dfs.name.dir configured. Of
> which one would be a remote location such as a nfs mount.
> So that even if the NN machine crashes on a whole you still have the fs
> image and edit log in nfs mount. This can be utilized for reconstructing
> the NN back again.
>
>
> Regards
> Bejoy KS
>
> Sent from remote device, Please excuse typos
> ------------------------------
> *From: * Panshul Whisper <[EMAIL PROTECTED]>
> *Date: *Mon, 14 Jan 2013 17:25:08 -0800
> *To: *<[EMAIL PROTECTED]>
> *ReplyTo: * [EMAIL PROTECTED]
> *Subject: *hadoop namenode recovery
>
> Hello,
>
> Is there a standard way to prevent the failure of Namenode crash in a
> Hadoop cluster?
> or what is the standard or best practice for overcoming the Single point
> failure problem of Hadoop.
>
> I am not ready to take chances on a production server with Hadoop 2.0
> Alpha release, which claims to have solved the problem. Are there any other
> things I can do to either prevent the failure or recover from the failure
> in a very short time.
>
> Thanking You,
>
> --
> Regards,
> Ouch Whisper
> 010101010101
>

--
Regards,
Ouch Whisper
010101010101