Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: backup node question


Copy link to this message
-
Re: backup node question
Harsh J 2013-04-07, 08:05
Yes, it need not keep an edits (transactions) stream locally cause
those are passed synchronously to the BackupNameNode, which persists
it on its behalf.

On Sun, Apr 7, 2013 at 1:21 PM, Lin Ma <[EMAIL PROTECTED]> wrote:
> Thanks Harsh,
>
> For your comments, "What it means is that the NameNode need not store
> anything locally", you mean Primary Name Node do not need to store
> checkpoint/journal locally, and only need to keep memory image up-to-date
> for edits?
>
> regards,
> Lin
>
>
> On Sun, Apr 7, 2013 at 3:31 PM, Harsh J <[EMAIL PROTECTED]> wrote:
>>
>> Hi Lin,
>>
>> My reply inline.
>>
>> On Sun, Apr 7, 2013 at 12:36 PM, Lin Ma <[EMAIL PROTECTED]> wrote:
>> > Hi guys,
>> >
>> > I am reading from this paper to learn about backup nodes
>> > (http://www.storageconference.org/2010/Papers/MSST/Shvachko.pdf),
>> >
>> > It is mentioned, "It contains all file system metadata information
>> > except
>> > for block locations. It can perform all operations of the regular
>> > NameNode
>> > that do not involve modification of the namespace or knowledge of block
>> > locations. ", what kinds of operations do not need knowledge of block
>> > locations?
>>
>> Operations that do not involve data reads or writes would not require
>> knowledge of block locations. Applying also the restriction of no
>> namespace mutation, an example would be listing directories and
>> looking up file information via FileStatus objects (perhaps the only
>> examples - its like a safemode but no reads either).
>>
>> > It is also mentioned, "Use of a BackupNode provides the option of
>> > running
>> > the NameNode without persistent storage, delegating responsibility for
>> > the
>> > namespace state persisting to the BackupNode.", what means "running the
>> > NameNode without persistent storage" and "delegating responsibility for
>> > the
>> > namespace state persisting"?
>>
>> What it means is that the NameNode need not store anything locally,
>> but can rely on the edits being stored at the BackupNameNode which
>> would continuously be receiving it. When restarted, it can grab a
>> current checkpoint from the BNN and boot up anywhere, since there's no
>> local storage requirement.
>>
>> --
>> Harsh J
>
>

--
Harsh J