Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # dev - Merging the HA branch to trunk - Wednesday, February 29th


+
Aaron T. Myers 2012-02-23, 02:24
+
Aaron T. Myers 2012-03-01, 00:00
+
Todd Lipcon 2012-03-01, 00:03
Copy link to this message
-
Re: Merging the HA branch to trunk - Wednesday, February 29th
Suresh Srinivas 2012-03-01, 01:42
I am +1 on merging this to trunk.

On Feb 29, 2012, at 4:03 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> +1 as well.
>
> My latest tests after applying the performance fixes indicate that
> there is no statistically significant performance regression between
> trunk and HA, even in tests designed to stress test the worst case
> code paths. The fixes should be committed today or tomorrow, but don't
> need to hold the merge up.
>
> -Todd
>
> On Wed, Feb 29, 2012 at 4:00 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>> Hello HDFS devs,
>>
>> The four JIRAs I mentioned below have all now been committed to the HA
>> branch. We've been running through the test plans posted on HDFS-1623 for
>> the last week.
>>
>> The only thing that's been discovered in the interim on the branch that
>> should perhaps be considered a blocker for the merge is the performance
>> issues Todd identified. I am of the opinion that we should proceed with the
>> merge to trunk anyway, despite the presence of these performance
>> regressions. Todd's done some good work on addressing those, which should
>> be committed in the next few days, so their presence on trunk should be
>> brief.
>>
>> If folks are amenable to the above, I'd like to do the merge to trunk
>> tomorrow, since it's getting a little late in the day today.
>>
>> Since this code change is a merge from a branch, we need three +1s to do
>> it. Clearly I'm +1.
>>
>> Thanks a lot,
>> Aaron
>>
>> --
>> Aaron T. Myers
>> Software Engineer, Cloudera
>>
>>
>>
>> On Wed, Feb 22, 2012 at 6:24 PM, Aaron T. Myers <[EMAIL PROTECTED]> wrote:
>>
>>> Hello HDFS devs,
>>>
>>> Work has largely stabilized on the HA-branch in the last few weeks. At
>>> this point the HA NN project is nearly feature-complete for manual
>>> failover. We've been running the full test suite nightly, and all automated
>>> tests have been passing, except for one known test failure which should be
>>> fixed shortly.
>>>
>>> I'd like to begin the process of merging this branch back to HDFS trunk.
>>> There are still several outstanding sub-JIRAs under the HDFS-1623 and
>>> HADOOP-7454 umbrella JIRAs, but most of these are either nice-to-haves or
>>> relate to supporting automatic failover. Once the branch is merged to
>>> trunk, work on these JIRAs can continue there.
>>>
>>> I've identified the following JIRAs which I think should be the only
>>> remaining blockers for merging to trunk:
>>>
>>> HDFS-2904 - Client support for getting delegation tokens in an HA cluster
>>> HDFS-2920 - Fix remaining TODOs in the code from HA. Mostly little cleanup
>>> stuff.
>>> HDFS-2958 - Sweep for remaining proxy construction which doesn't go
>>> through failover path
>>> HDFS-2979 - Balancer should use logical URI for creating failover proxy
>>> (will fix the only current test failure)
>>>
>>> All of these JIRAs should be fixed in the next few days.
>>>
>>> I propose that, unless more blocker issues are discovered in the interim,
>>> we merge this branch to trunk one week from today, i.e. Wednesday, February
>>> 29th. During this time we will also execute the test plans described in the
>>> test documents attached to HDFS-1623 to try to identify any regressions or
>>> performance issues in the branch. If you plan to review the code changes or
>>> the test plan, I ask that you please do so as soon as possible.
>>>
>>> Feedback is certainly welcome on this plan.
>>>
>>> Thanks a lot,
>>> Aaron
>>>
>>> --
>>> Aaron T. Myers
>>> Software Engineer, Cloudera
>>>
>>>
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
+
Vinod Kumar Vavilapalli 2012-03-01, 03:52
+
Tsz Wo Sze 2012-03-01, 22:10
+
Aaron T. Myers 2012-03-01, 22:39
+
Aaron T. Myers 2012-03-02, 05:01
+
Tsz Wo Sze 2012-03-01, 22:24
+
Aaron T. Myers 2012-03-01, 22:51
+
Tsz Wo Sze 2012-03-01, 00:32
+
Todd Lipcon 2012-03-01, 00:36
+
Tsz Wo Sze 2012-03-01, 01:01
+
Todd Lipcon 2012-03-01, 01:04
+
Tsz Wo Sze 2012-03-01, 01:17
+
Eli Collins 2012-02-23, 03:37
+
Jitendra Pandey 2012-02-23, 06:05
+
Arun C Murthy 2012-02-23, 06:12
+
Suresh Srinivas 2012-02-23, 18:23
+
Todd Lipcon 2012-02-23, 18:27
+
Todd Lipcon 2012-02-24, 02:08