Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # dev - Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk


+
sanjay Radia 2012-09-28, 19:06
+
Todd Lipcon 2012-09-28, 22:02
+
sanjay Radia 2012-10-01, 17:55
+
Todd Lipcon 2012-10-09, 00:46
+
Suresh Srinivas 2012-10-09, 01:01
+
Todd Lipcon 2012-10-09, 01:20
+
Suresh Srinivas 2012-10-09, 02:21
+
Eli Collins 2012-10-09, 04:11
+
sanjay Radia 2012-10-09, 20:44
+
sanjay Radia 2012-10-11, 02:17
+
Suresh Srinivas 2012-10-11, 04:10
+
Todd Lipcon 2012-10-11, 08:12
+
Andrew Purtell 2012-10-09, 03:03
+
Suresh Srinivas 2012-10-09, 03:32
+
Andrew Purtell 2012-10-09, 03:09
+
Eli Collins 2012-10-09, 01:48
Copy link to this message
-
Re: [VOTE] Merge HDFS-3077 (QuorumJournalManager) branch to trunk
sanjay Radia 2012-09-28, 19:03

 I am in favor of 3077  being part of HDFS.
   - It is one of several  way of configuring HDFS HA (shared storage is an option or if we extend the BackupNode a little, then it will be  another option for HA.)
  -  I believe I can use part of 3092/3077  to move journals to the Datanodes down the road.
- I have a hard time seeing 3077 having independent value outside of HDFS unless a fair amount of work is added to 3077.
- I do agree with Konstantin that this is fairly complex work;  by merging it in we are adding a fair amount  of future maintenance work. I am reviewing the design and code to see if we can simplify parts of this.
On a separate note - response to some of the comments in this thread.
- Even though I am in favor of this becoming part of HDFS, I don't buy the argument that HDFS HA should not depend on external projects.  
    - HDFS HA needs ZK which is an external project. I also think it fine for folks to use Linux HA  - there to be a blueprint for HDFS HA with Linux HA for those who choose to use it.

I  never bought this argument in the case of BK either.
Wrt to BK I feel the HDFS team had done a  NIH. I believe we could have worked with the BK team to add/modify it as needed. The BK team was willing.
We originally started with a simpler design for the journal daemon/node and kept on finding bugs in the design and then ended up reinventing some of the BK work.
Yes i have read todd's arguments and have debated this with suresh at great length -  reasonable folks disagree at times )

To summarize - i am in favor of this being part of HDFS and am reviewing the design and code.
sanjay
On Sep 26, 2012, at 2:32 PM, Stack wrote:

> On Tue, Sep 25, 2012 at 11:21 PM, Konstantin Shvachko
> <[EMAIL PROTECTED]> wrote:
>> I think this is a great work, Todd.
>> And I think we should not merge it into trunk or other branches.
>> As I suggested earlier on this list I think this should be spinned off
>> as a separate project or a subproject.
>>
>
> I'd be -1 on that.
>
> Users shouldn't have to go elsewhere to get a fix for SPOF.
>
> St.Ack
+
Stack 2012-09-27, 16:59
+
Todd Lipcon 2012-09-27, 20:29
+
Suresh Srinivas 2012-09-27, 16:47
+
Andrew Purtell 2012-09-27, 09:29