Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

Copy link to this message
Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk
Sorry for the slowness in response - I've been in bed with a fever the
past couple days and only sporadically on email. I'll respond to a few
different points made in the above thread here:

Konst> We keep growing the HDFS umbrella with competing technologies
Konst> (http/web HDFS as an example) within it.

I suppose these are "competing technologies" in the sense that there
are multiple options to solve the same goal: NFS shared storage,
BookKeeper-based storage, and QJM-based storage. On the other hand,
they are quite distinct in my view:
- NFS shared storage: this simply re-uses existing code
(FileJournalManager) that we already have for the non-HA case. It has
several downsides for use in an HA cluster (see the HDFS-3077 design
doc for discussion of them).
- BK storage: this is advantageous for some people who already run
BookKeeper in their datacenter. However, I think for those not already
running it, it adds much complexity, and lacks several advantages of
the QJM - again, see the design discussion on HDFS-3077 for details.
The tldr version is: Hadoop-consistent metrics, security, IPC, and
- QJM storage: I won't expand on the advantages here, please refer to
the design doc.

So, if they fulfill the same goal, why let them coexist in the codebase?

I would argue that this is one of those things that often happens in a
community-developed project: different users or vendors may have
slightly different requirements, and therefore prefer different
solutions. Some people already have an HA NFS filer and run all their
HA using NFS, so the NFS shared storage seems "safest" to them. Others
already run BookKeeper, so they feel most comfortable with that
approach (or they want to use BookKeeper for other replicated logs).
For me and a lot of our customers, QJM seems to be the best approach.

Rather than duking it out and trying to declare one the "winner", our
answer has generally been "make it pluggable, and people can pick the
implementation that's right for them". If at some point later, it
turns out that everyone's using the same plug-in, by all means we
should consider ejecting the others, eg to github. But while there are
several active committers who have pledged that they will maintain a
solution, it seems reasonable to keep it as part of HDFS core. In this
case, I'm certainly planning on maintaining it, and it seems like
others on the list are on board as well.

Konst> Which makes the project harder to stabilize and release.
Konst> Not touching MR/Yarn here.
Konst> My concern is that if we add another 6000 lines of code to Hadoop-2,
Konst> it will take yet another x months for stabilization.

As you pointed out in your first email, the new code is mostly
stand-alone. The changes to the core NN are really quite small, and
actually benefit all of the storage implementations (eg enhancing the
web UI to display non-file JMs). So, if you don't choose to add a
qjournal:// URI on your cluster, it won't affect your stability at

On the other hand, I happen to believe it is as stable as the rest of
Hadoop. Several committers, as well as some folks in the community,
and our internal QA team have been testing it for a couple months now,
with extensive fault injection, real workloads, etc, and it has held
up nicely. Stability is of course subjective in some sense, so if you
want to think of it as unstable owing to the fact that it's new code,
I'll respect that. But I don't think you can reject new code from a
project just because it's new code.

Konst> While it is not clear why people cannot just use NFS filers for
shared storage,
Konst> as you originally designed.

This was always seen as a step along the way to the final solution. As
several people have said, NFS filers are not available in every
organization. It was a useful point along the way, as it provided a
working production-ready HA solution several months back, whereas
waiting for a non-NFS solution would have delayed it quite a bit. For
reference, check out Sanjay and Aaron's presentation from Hadoop World
last year: http://www.slideshare.net/cloudera/hadoop-world-2011-hdfs-name-node-high-availablity-aaron-myers-cloudera-sanjay-radia-hortonworks
 ("Other options to share NN metadata" under "Future Work")

Konst> Don't understand your argument. Else where?
Konst> One way or another users will be talking to Todd.

Not sure what you meant by this. Sure, I wrote a lot of the code, but
it's a contribution to the community, and in no way do I have a
monopoly on helping people use it (I certainly hope I don't!) The
branch to be merged includes full docs on how to get running, and I
certainly hope that other Hadoop vendors will ship and recommend this
option too. And those who choose to consume Hadoop directly from
Apache, rather than through a vendor, should be able to use it equally
well, without patching together bits and pieces from a number of

On Thu, Sep 27, 2012 at 9:59 AM, Stack <[EMAIL PROTECTED]> wrote:

Todd Lipcon
Software Engineer, Cloudera
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB