-Re: One NameNode keeps Rolling Edit Log on HDFS Federation
YouPeng Yang 2013-02-25, 01:06
Thank you for your reply.
I need to time to digest the above stuff.
2013/2/25 Harsh J <[EMAIL PROTECTED]>
> I don't see how SPOF comes into the picture when HA is already present
> in the releases that also carry Federation and each NN (federated or
> non) can be assigned further Standby-NN roles. At this point we can
> stop using the word "SPOF" completely for HDFS. Would do great good
> for avoiding further FUD around this :)
> The federated NameNodes are namespace-divided; and typically I think
> users may use it to divide applications into using different
> namespaces each, while using the same set of compute and data nodes.
> So if HA isn't deployed, an application may be lost but not "portion
> of the cluster", since DNs will remain unaffected and so would the
> other namespaces.
> On Sun, Feb 24, 2013 at 11:23 PM, Michel Segel
> <[EMAIL PROTECTED]> wrote:
> > I think part of the confusion stems from the fact that federation of
> name nodes only splits the very large cluster in to smaller portions of the
> same cluster. If you lose a federated name node, you only lose a portion of
> the cluster not the whole thing. So now instead of one SPOF, you have two
> > The advantage of Federation is that you reduce the amount of memory
> required for the NN.
> > Sent from a remote device. Please excuse any typos...
> > Mike Segel
> > On Feb 24, 2013, at 10:48 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> >> Hi,
> >> Federated namenodes are independent of one another (except that they
> >> both get reports from all the/common DNs in the cluster). It is
> >> natural to see one roll its edit logs based on its own rate of
> >> metadata growth, as compared to the other. Their edits, image, etc.
> >> everything is independent - they also do not know about/talk to each
> >> other.
> >> Given the above, I'd say (1) is true and (2) is invalid, and (3) can
> >> be found in PDFs attached to
> >> https://issues.apache.org/jira/browse/HDFS-1052.
> >> On Sun, Feb 24, 2013 at 8:39 PM, YouPeng Yang <
> [EMAIL PROTECTED]> wrote:
> >>> Hi All
> >>> I'm testing the HDFS Federation.I have 2 namenodes in my cluster.
> >>> I find that it is Rolling Edit Log continuously on one
> >>> the other one changes nothing.
> >>> My question :
> >>> 1. Is it the right situation.
> >>> 2. I have thought that the two namenodes shoud keep concurrency.
> >>> why they get differences.
> >>> 3. is there any advanced docs about HDFS Federation.
> >>> thanks.
> >>> Regards.
> >> --
> >> Harsh J
> Harsh J