Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> NameNode failure and recovery!


Copy link to this message
-
Re: NameNode failure and recovery!
Thats also doable , reducing the checkpoint period would also have have
some amount of edit log loss and how short should be the checkpoint
interval has to be evaluated.I think the good way to go , in case HA is not
doable is SNN and secondary storage NFS.

Thanks,
Rahul
On Thu, Apr 4, 2013 at 12:19 AM, shashwat shriparv <
[EMAIL PROTECTED]> wrote:

> If you are not in position to go for HA just keep your checkpoint period
> shorter to have recent data recoverable from SNN.
>
> and you always have a option
> hadoop namenode -recover
> try this on testing cluster and get versed to it.
>
> and take backup of image at some solid state storage.
>
>
>
> ∞
> Shashwat Shriparv
>
>
>
> On Wed, Apr 3, 2013 at 9:56 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>
>> There is a 3rd, most excellent way: Use HDFS's own HA, see
>>
>> http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-site/HDFSHighAvailabilityWithQJM.html
>> :)
>>
>> On Wed, Apr 3, 2013 at 8:10 PM, Rahul Bhattacharjee
>> <[EMAIL PROTECTED]> wrote:
>> > Hi all,
>> >
>> > I was reading about Hadoop and got to know that there are two ways to
>> > protect against the name node failures.
>> >
>> > 1) To write to a nfs mount along with the usual local disk.
>> >  -or-
>> > 2) Use secondary name node. In case of failure of NN , the SNN can take
>> in
>> > charge.
>> >
>> > My questions :-
>> >
>> > 1) SNN is always lagging , so when SNN becomes primary in event of a NN
>> > failure ,  then the edits which have not been merged into the image file
>> > would be lost , so the system of SNN would not be consistent with the NN
>> > before its failure.
>> >
>> > 2) Also I have read that other purpose of SNN is to periodically merge
>> the
>> > edit logs with the image file. In case a setup goes with option #1
>> (writing
>> > to NFS, no SNN) , then who does this merging.
>> >
>> > Thanks,
>> > Rahul
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB