Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk


Copy link to this message
-
Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk

 I am in favor of 3077  being part of HDFS.
   - It is one of several  way of configuring HDFS HA (shared storage is an option or if we extend the BackupNode a little, then it will be  another option for HA.)
  -  I believe I can use part of 3092/3077  to move journals to the Datanodes down the road.
- I have a hard time seeing 3077 having independent value outside of HDFS unless a fair amount of work is added to 3077.
- I do agree with Konstantin that this is fairly complex work;  by merging it in we are adding a fair amount  of future maintenance work. I am reviewing the design and code to see if we can simplify parts of this.
On a separate note - response to some of the comments in this thread.
- Even though I am in favor of this becoming part of HDFS, I don't buy the argument that HDFS HA should not depend on external projects.  
    - HDFS HA needs ZK which is an external project. I also think it fine for folks to use Linux HA  - there to be a blueprint for HDFS HA with Linux HA for those who choose to use it.

I  never bought this argument in the case of BK either.
Wrt to BK I feel the HDFS team had done a  NIH. I believe we could have worked with the BK team to add/modify it as needed. The BK team was willing.
We originally started with a simpler design for the journal daemon/node and kept on finding bugs in the design and then ended up reinventing some of the BK work.
Yes i have read todd's arguments and have debated this with suresh at great length -  reasonable folks disagree at times )

To summarize - i am in favor of this being part of HDFS and am reviewing the design and code.
sanjay
On Sep 26, 2012, at 2:32 PM, Stack wrote:

> On Tue, Sep 25, 2012 at 11:21 PM, Konstantin Shvachko
> <[EMAIL PROTECTED]> wrote:
>> I think this is a great work, Todd.
>> And I think we should not merge it into trunk or other branches.
>> As I suggested earlier on this list I think this should be spinned off
>> as a separate project or a subproject.
>>
>
> I'd be -1 on that.
>
> Users shouldn't have to go elsewhere to get a fix for SPOF.
>
> St.Ack