Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk


Copy link to this message
-
Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

 I am in favor of 3077  being part of HDFS.
   - It is one of several  way of configuring HDFS HA (shared storage is an option or if we extend the BackupNode a little, then it will be  another option for HA.)
  -  I believe I can use part of 3092/3077  to move journals to the Datanodes down the road.
- I have a hard time seeing 3077 having independent value outside of HDFS unless a fair amount of work is added to 3077.
- I do agree with Konstantin that this is fairly complex work;  by merging it in we are adding a fair amount  of future maintenance work. I am reviewing the design and code to see if we can simplify parts of this.
On a separate note - response to some of the comments in this thread.
- Even though I am in favor of this becoming part of HDFS, I don't buy the argument that HDFS HA should not depend on external projects.  
    - HDFS HA needs ZK which is an external project. I also think it fine for folks to use Linux HA  - there to be a blueprint for HDFS HA with Linux HA for those who choose to use it.

I  never bought this argument in the case of BK either.
Wrt to BK I feel the HDFS team had done a  NIH. I believe we could have worked with the BK team to add/modify it as needed. The BK team was willing.
We originally started with a simpler design for the journal daemon/node and kept on finding bugs in the design and then ended up reinventing some of the BK work.
Yes i have read todd's arguments and have debated this with suresh at great length -  reasonable folks disagree at times )

To summarize - i am in favor of this being part of HDFS and am reviewing the design and code.
sanjay
On Sep 26, 2012, at 2:32 PM, Stack wrote:

> On Tue, Sep 25, 2012 at 11:21 PM, Konstantin Shvachko
> <[EMAIL PROTECTED]> wrote:
>> I think this is a great work, Todd.
>> And I think we should not merge it into trunk or other branches.
>> As I suggested earlier on this list I think this should be spinned off
>> as a separate project or a subproject.
>>
>
> I'd be -1 on that.
>
> Users shouldn't have to go elsewhere to get a fix for SPOF.
>
> St.Ack
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB