Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # dev - Re-swizzle 2.3


Copy link to this message
-
Re: Re-swizzle 2.3
Sandy Ryza 2014-02-07, 01:15
+1 to reverting those JIRAs from branch-2.3.  As YARN-1689 is fixing a
problem caused by YARN-1493 I think we can revert it in branch-2.3 as well.

I think we should leave them in branch-2 for now.  We can revert if 2.4 is
imminent and they're holding it up, but hopefully the issues they caused
will be fixed by then.

-Sandy
On Thu, Feb 6, 2014 at 5:07 PM, Alejandro Abdelnur <[EMAIL PROTECTED]>wrote:

> Thanks Robert,
>
> All,
>
> So it seems that YARN-1493 and YARN-1490 are introducing serious
> regressions.
>
> I would propose to revert them and the follow up JIRAs from the 2.3 branch
> and keep working on them on trunk/branch-2 until the are stable (I would
> even prefer reverting them from branch-2 not to block a 2.4 if they are not
> ready in time).
>
> As I've mentioned before, the list of JIRAs to revert were:
>
> YARN-1493
> YARN-1490
> YARN-1166
> YARN-1041
> YARN-1566
>
> Plus 2 additional JIRAs committed since my email on this issue 2 days ago:
>
> *YARN-1661
> *YARN-1689 (not sure if this JIRA is related in functionality to the
> previous ones but it is creating conflicts).
>
> I think we should hold on continuing work on top of something that is
> broken until the broken stuff is fixed.
>
> Quoting Arun, "Committers - Henceforth, please use extreme caution while
> committing to branch-2.3. Please commit *only* blockers to 2.3."
>
> YARN-1661 & YARN-1689 are not blockers.
>
> Unless there are objections, I'll revert all these JIRAs from branch-2.3
> tomorrow around noon and I'll update fixedVersion in the JIRAs.
>
> I'm inclined to revert them from branch-2 as well.
>
> Thoughts?
>
> Thanks.
>
>
> On Thu, Feb 6, 2014 at 3:54 PM, Robert Kanter <[EMAIL PROTECTED]>
> wrote:
>
> > I think we should revert YARN-1490 from Hadoop 2.3 branch.  I think it
> was
> > causing some strange behavior in the Oozie unit tests:
> >
> > Basically, we use a single MiniMRCluster and MiniDFSCluster across all
> unit
> > tests in a module.  With YARN-1490 we saw that, regardless of test order,
> > the last few tests would timeout waiting for an MR job to finish; on
> slower
> > machines, the entire test suite would timeout.  Through some digging, I
> > found that we were getting a ton of "Connection refused" Exceptions on
> > LeaseRenewer talking to the NN and a few on the AM talking to the RM.
> >
> > After a bunch of investigation, I found that the problem went away once
> > YARN-1490 was removed.  Though I couldn't figure out the exact problem.
> >  Even though this occurred in unit tests, it does make me concerned that
> it
> > could indicate some bigger issue in a long-running real cluster (where
> > everything isn't running on the same machine) that we haven't seen yet.
> >
> >
> >
> > On Thu, Feb 6, 2014 at 3:06 PM, Karthik Kambatla <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I have marked MAPREDUCE-5744 a blocker for 2.3. Committing it shortly.
> > Will
> > > pull it out of branch-2.3 if anyone objects.
> > >
> > >
> > > On Thu, Feb 6, 2014 at 2:04 PM, Arpit Agarwal <
> [EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Merged HADOOP-10273 to branch-2.3 as r1565456.
> > > >
> > > >
> > > > On Wed, Feb 5, 2014 at 4:49 PM, Arpit Agarwal <
> > [EMAIL PROTECTED]
> > > > >wrote:
> > > >
> > > > > IMO HADOOP-10273 (Fix 'mvn site') should be included in 2.3.
> > > > >
> > > > > I will merge it to branch-2.3 tomorrow PST if no one disagrees.
> > > > >
> > > > >
> > > > > On Tue, Feb 4, 2014 at 5:03 PM, Alejandro Abdelnur <
> > [EMAIL PROTECTED]
> > > > >wrote:
> > > > >
> > > > >> IMO YARN-1577 is a blocker, it is breaking unmanaged AMs in a very
> > odd
> > > > >> ways
> > > > >> (to the point it seems un-deterministic).
> > > > >>
> > > > >> I'd say eiher YARN-1577 is fixed or we revert
> > > > >> YARN-1493/YARN-1490/YARN-1166/YARN-1041/YARN-1566 (almost clean
> > > reverts)
> > > > >> from Hadoop 2.3 branch before doing the release.
> > > > >>
> > > > >>
> > > > >> I've verified that after reverting those JIRAs things work fine