Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - backup node question


+
Lin Ma 2013-04-07, 07:06
+
Harsh J 2013-04-07, 07:31
+
Lin Ma 2013-04-07, 07:51
+
Lin Ma 2013-04-07, 08:15
Copy link to this message
-
Re: backup node question
Harsh J 2013-04-07, 13:02
BackupNameNode is not present in the maintenance 1.x releases, it is a
feature added to a higher version; you can try it out in 2.x today if
you wish to.

On Sun, Apr 7, 2013 at 3:12 PM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
> Hi Harsh,
> Do you mean BackupNameNode is Secondary NameNode in Hadoop1.x?
>
>
> On Sun, Apr 7, 2013 at 4:05 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> Yes, it need not keep an edits (transactions) stream locally cause
>> those are passed synchronously to the BackupNameNode, which persists
>> it on its behalf.
>>
>> On Sun, Apr 7, 2013 at 1:21 PM, Lin Ma <[EMAIL PROTECTED]> wrote:
>> > Thanks Harsh,
>> >
>> > For your comments, "What it means is that the NameNode need not store
>> > anything locally", you mean Primary Name Node do not need to store
>> > checkpoint/journal locally, and only need to keep memory image
>> > up-to-date
>> > for edits?
>> >
>> > regards,
>> > Lin
>> >
>> >
>> > On Sun, Apr 7, 2013 at 3:31 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>> >>
>> >> Hi Lin,
>> >>
>> >> My reply inline.
>> >>
>> >> On Sun, Apr 7, 2013 at 12:36 PM, Lin Ma <[EMAIL PROTECTED]> wrote:
>> >> > Hi guys,
>> >> >
>> >> > I am reading from this paper to learn about backup nodes
>> >> > (http://www.storageconference.org/2010/Papers/MSST/Shvachko.pdf),
>> >> >
>> >> > It is mentioned, "It contains all file system metadata information
>> >> > except
>> >> > for block locations. It can perform all operations of the regular
>> >> > NameNode
>> >> > that do not involve modification of the namespace or knowledge of
>> >> > block
>> >> > locations. ", what kinds of operations do not need knowledge of block
>> >> > locations?
>> >>
>> >> Operations that do not involve data reads or writes would not require
>> >> knowledge of block locations. Applying also the restriction of no
>> >> namespace mutation, an example would be listing directories and
>> >> looking up file information via FileStatus objects (perhaps the only
>> >> examples - its like a safemode but no reads either).
>> >>
>> >> > It is also mentioned, "Use of a BackupNode provides the option of
>> >> > running
>> >> > the NameNode without persistent storage, delegating responsibility
>> >> > for
>> >> > the
>> >> > namespace state persisting to the BackupNode.", what means "running
>> >> > the
>> >> > NameNode without persistent storage" and "delegating responsibility
>> >> > for
>> >> > the
>> >> > namespace state persisting"?
>> >>
>> >> What it means is that the NameNode need not store anything locally,
>> >> but can rely on the edits being stored at the BackupNameNode which
>> >> would continuously be receiving it. When restarted, it can grab a
>> >> current checkpoint from the BNN and boot up anywhere, since there's no
>> >> local storage requirement.
>> >>
>> >> --
>> >> Harsh J
>> >
>> >
>>
>>
>>
>> --
>> Harsh J
>
>

--
Harsh J