Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Read namenode data from primary memory


+
Aditya exalter 2013-05-14, 06:37
+
Harsh J 2013-05-14, 07:46
+
Ling Kun 2013-05-14, 06:48
+
Sandeep Nemuri 2013-05-14, 06:57
+
Nitin Pawar 2013-05-14, 07:01
+
Sandeep Nemuri 2013-05-14, 07:21
+
Nitin Pawar 2013-05-14, 07:23
+
Sandeep Nemuri 2013-05-14, 07:27
Copy link to this message
-
Re: Read namenode data from primary memory
Nitin Pawar 2013-05-14, 07:43
if you want to use savenamespace then yes.

Normally to have a backup copy of the FSImage, previously there was a
secondary namenode and now there is checkpoint NN. SNN was deprecated but
later it was undeprecated

the sole purpose of checkpoint NN is to have an rolling copy of the NN
based on interval you configure.
Checkpoint Node is the process which is actually responsible for creating
periodic check points. It downloads fsimage and log edits from the active
NameNode, merges them locally, and uploads the new image back to the active
NameNode.

It is also a good practice that you configure multiple directories for
saving fsimage and editlogs on different disks. (From cloudera blog: By
storing two copies of the edit log and FSImage, on two separate hard disks,
a good system administrator can avoid bringing down the NameNode if one of
those disks fails.)
On Tue, May 14, 2013 at 12:57 PM, Sandeep Nemuri <[EMAIL PROTECTED]>wrote:

> Done Lets say 8hrs !!
> now every 8hrs do we need to go into safemode ?
>
>
> On Tue, May 14, 2013 at 12:53 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>
>> every 5 minutes seems too much.
>>
>> why not do a daily or say 8 hours basis
>>
>>
>> On Tue, May 14, 2013 at 12:51 PM, Sandeep Nemuri <[EMAIL PROTECTED]>wrote:
>>
>>> I Agree with you Nitin ..
>>> My Requirement is i want to copy the edits and fsimage from memory for
>>> every 5mins. How does this works ?
>>>
>>>
>>> On Tue, May 14, 2013 at 12:31 PM, Nitin Pawar <[EMAIL PROTECTED]>wrote:
>>>
>>>> if you do not put namenode in safemode that means your namenode can
>>>> still accept request to write or delete files from hdfs. This may cause
>>>> your snapshot to be a problematic in case you want to restore from it.
>>>> Safemode is just an ensuring point that you have a standstill image to
>>>> snapshot.
>>>>
>>>>
>>>>
>>>>
>>>> On Tue, May 14, 2013 at 12:27 PM, Sandeep Nemuri <[EMAIL PROTECTED]>wrote:
>>>>
>>>>> Withuot going to safemode can't we get the Namespace ??
>>>>>
>>>>>
>>>>> On Tue, May 14, 2013 at 12:18 PM, Ling Kun <[EMAIL PROTECTED]>wrote:
>>>>>
>>>>>> Hi Aditya:
>>>>>>    This command line may help you:
>>>>>>
>>>>>>      hadoop dfsadmin --saveNamespace
>>>>>>
>>>>>>      you should firstly perform it in safe mode.
>>>>>>
>>>>>>     The corresponding design and implementation is described here:
>>>>>> https://issues.apache.org/jira/browse/HADOOP-4826
>>>>>>
>>>>>>
>>>>>> yours,
>>>>>> Ling Kun
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Tue, May 14, 2013 at 2:37 PM, Aditya exalter <
>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> Hi,
>>>>>>>
>>>>>>>      How to get the namenode data(edits and fsimage) from the main
>>>>>>> memory(RAM)
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> http://www.lingcc.com
>>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> --Regards
>>>>>   Sandeep Nemuri
>>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Nitin Pawar
>>>>
>>>
>>>
>>>
>>> --
>>> --Regards
>>>   Sandeep Nemuri
>>>
>>
>>
>>
>> --
>> Nitin Pawar
>>
>
>
>
> --
> --Regards
>   Sandeep Nemuri
>

--
Nitin Pawar
+
Leonid Fedotov 2013-05-14, 07:51
+
Leonid Fedotov 2013-05-14, 07:43