Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> NameNode failure and recovery!


+
Rahul Bhattacharjee 2013-04-03, 14:40
+
Azuryy Yu 2013-04-03, 15:08
+
Rahul Bhattacharjee 2013-04-03, 14:42
+
Mohammad Tariq 2013-04-03, 14:57
Copy link to this message
-
Re: NameNode failure and recovery!
@Vijay : We seem to be in 100% sync though :)

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Wed, Apr 3, 2013 at 8:27 PM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Hello Rahul,
>
>       It's always better to have both 1 and 2 together. One common
> misconception is that SNN is a backup of the NN, which is wrong. SNN is a
> helper node to the NN. In case of any failure SNN is not gonna take up the
> NN spot.
>
> Yes, we can't guarantee that the SNN fsimage replica will always be up to
> date. And when you are writing the metadata on a filer or NFS, you are just
> creating an additional copy of the metadata. Don't mistake it with SNN.
> When you specify value of your "dfs.name.dir" property as a comma separated
> list, which is localFS+NFS, you are just making sure that even if something
> goes wrong with the localFS, your metadata is still same in the NFS.
>
> But, it is still better to have the SNN in a separate machine. But you can
> never rely 100% on SNN, because of the fact you have already mentioned.
> It'll not be in 100% sync.
>
>
>
> Warm Regards,
> Tariq
> https://mtariq.jux.com/
> cloudfront.blogspot.com
>
>
> On Wed, Apr 3, 2013 at 8:12 PM, Rahul Bhattacharjee <
> [EMAIL PROTECTED]> wrote:
>
>> Or both the options are used together. NFS + SNN ?
>>
>>
>>
>>  On Wed, Apr 3, 2013 at 8:10 PM, Rahul Bhattacharjee <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi all,
>>>
>>> I was reading about Hadoop and got to know that there are two ways to
>>> protect against the name node failures.
>>>
>>> 1) To write to a nfs mount along with the usual local disk.
>>>  -or-
>>> 2) Use secondary name node. In case of failure of NN , the SNN can take
>>> in charge.
>>>
>>> My questions :-
>>>
>>> 1) SNN is always lagging , so when SNN becomes primary in event of a NN
>>> failure ,  then the edits which have not been merged into the image file
>>> would be lost , so the system of SNN would not be consistent with the NN
>>> before its failure.
>>>
>>> 2) Also I have read that other purpose of SNN is to periodically merge
>>> the edit logs with the image file. In case a setup goes with option #1
>>> (writing to NFS, no SNN) , then who does this merging.
>>>
>>> Thanks,
>>> Rahul
>>>
>>>
>>>
>>
>
+
shashwat shriparv 2013-04-03, 18:49
+
Rahul Bhattacharjee 2013-04-04, 03:12
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB