Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Re: backup node question


Copy link to this message
-
Re: backup node question
Azuryy Yu 2013-04-07, 14:12
oh, got it. you are a good guy.

--Send from my Sony mobile.
On Apr 7, 2013 10:11 PM, "Harsh J" <[EMAIL PROTECTED]> wrote:

> StandbyNameNode is the term we use to refer to a NameNode in HA that
> is currently not the active one (i.e. its state is 'Standby'). Its not
> a special type of daemon (i.e. it just runs the NameNode service),
> just a naming convention.
>
> On Sun, Apr 7, 2013 at 7:31 PM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
> > I am confused. Hadoopv2 has NN SNN DN JN(journal node), so whats
> > Standby Namenode?
> >
> > --Send from my Sony mobile.
> >
> > On Apr 7, 2013 9:03 PM, "Harsh J" <[EMAIL PROTECTED]> wrote:
> >>
> >> BackupNameNode is not present in the maintenance 1.x releases, it is a
> >> feature added to a higher version; you can try it out in 2.x today if
> >> you wish to.
> >>
> >> On Sun, Apr 7, 2013 at 3:12 PM, Azuryy Yu <[EMAIL PROTECTED]> wrote:
> >> > Hi Harsh,
> >> > Do you mean BackupNameNode is Secondary NameNode in Hadoop1.x?
> >> >
> >> >
> >> > On Sun, Apr 7, 2013 at 4:05 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> >> >>
> >> >> Yes, it need not keep an edits (transactions) stream locally cause
> >> >> those are passed synchronously to the BackupNameNode, which persists
> >> >> it on its behalf.
> >> >>
> >> >> On Sun, Apr 7, 2013 at 1:21 PM, Lin Ma <[EMAIL PROTECTED]> wrote:
> >> >> > Thanks Harsh,
> >> >> >
> >> >> > For your comments, "What it means is that the NameNode need not
> store
> >> >> > anything locally", you mean Primary Name Node do not need to store
> >> >> > checkpoint/journal locally, and only need to keep memory image
> >> >> > up-to-date
> >> >> > for edits?
> >> >> >
> >> >> > regards,
> >> >> > Lin
> >> >> >
> >> >> >
> >> >> > On Sun, Apr 7, 2013 at 3:31 PM, Harsh J <[EMAIL PROTECTED]>
> wrote:
> >> >> >>
> >> >> >> Hi Lin,
> >> >> >>
> >> >> >> My reply inline.
> >> >> >>
> >> >> >> On Sun, Apr 7, 2013 at 12:36 PM, Lin Ma <[EMAIL PROTECTED]> wrote:
> >> >> >> > Hi guys,
> >> >> >> >
> >> >> >> > I am reading from this paper to learn about backup nodes
> >> >> >> > (http://www.storageconference.org/2010/Papers/MSST/Shvachko.pdf
> ),
> >> >> >> >
> >> >> >> > It is mentioned, "It contains all file system metadata
> information
> >> >> >> > except
> >> >> >> > for block locations. It can perform all operations of the
> regular
> >> >> >> > NameNode
> >> >> >> > that do not involve modification of the namespace or knowledge
> of
> >> >> >> > block
> >> >> >> > locations. ", what kinds of operations do not need knowledge of
> >> >> >> > block
> >> >> >> > locations?
> >> >> >>
> >> >> >> Operations that do not involve data reads or writes would not
> >> >> >> require
> >> >> >> knowledge of block locations. Applying also the restriction of no
> >> >> >> namespace mutation, an example would be listing directories and
> >> >> >> looking up file information via FileStatus objects (perhaps the
> only
> >> >> >> examples - its like a safemode but no reads either).
> >> >> >>
> >> >> >> > It is also mentioned, "Use of a BackupNode provides the option
> of
> >> >> >> > running
> >> >> >> > the NameNode without persistent storage, delegating
> responsibility
> >> >> >> > for
> >> >> >> > the
> >> >> >> > namespace state persisting to the BackupNode.", what means
> >> >> >> > "running
> >> >> >> > the
> >> >> >> > NameNode without persistent storage" and "delegating
> >> >> >> > responsibility
> >> >> >> > for
> >> >> >> > the
> >> >> >> > namespace state persisting"?
> >> >> >>
> >> >> >> What it means is that the NameNode need not store anything
> locally,
> >> >> >> but can rely on the edits being stored at the BackupNameNode which
> >> >> >> would continuously be receiving it. When restarted, it can grab a
> >> >> >> current checkpoint from the BNN and boot up anywhere, since
> there's
> >> >> >> no
> >> >> >> local storage requirement.
> >> >> >>
> >> >> >> --
> >> >> >> Harsh J
> >> >> >
> >> >> >
> >> >>
> >> >>
> >> >>
> >> >> --
> >> >> Harsh J
> >> >
> >> >
> >>
> >>
> >