Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Re-swizzle 2.3


Copy link to this message
-
Re: Re-swizzle 2.3
I think we should revert YARN-1490 from Hadoop 2.3 branch.  I think it was
causing some strange behavior in the Oozie unit tests:

Basically, we use a single MiniMRCluster and MiniDFSCluster across all unit
tests in a module.  With YARN-1490 we saw that, regardless of test order,
the last few tests would timeout waiting for an MR job to finish; on slower
machines, the entire test suite would timeout.  Through some digging, I
found that we were getting a ton of "Connection refused" Exceptions on
LeaseRenewer talking to the NN and a few on the AM talking to the RM.

After a bunch of investigation, I found that the problem went away once
YARN-1490 was removed.  Though I couldn't figure out the exact problem.
 Even though this occurred in unit tests, it does make me concerned that it
could indicate some bigger issue in a long-running real cluster (where
everything isn't running on the same machine) that we haven't seen yet.

On Thu, Feb 6, 2014 at 3:06 PM, Karthik Kambatla <[EMAIL PROTECTED]> wrote:

> I have marked MAPREDUCE-5744 a blocker for 2.3. Committing it shortly. Will
> pull it out of branch-2.3 if anyone objects.
>
>
> On Thu, Feb 6, 2014 at 2:04 PM, Arpit Agarwal <[EMAIL PROTECTED]
> >wrote:
>
> > Merged HADOOP-10273 to branch-2.3 as r1565456.
> >
> >
> > On Wed, Feb 5, 2014 at 4:49 PM, Arpit Agarwal <[EMAIL PROTECTED]
> > >wrote:
> >
> > > IMO HADOOP-10273 (Fix 'mvn site') should be included in 2.3.
> > >
> > > I will merge it to branch-2.3 tomorrow PST if no one disagrees.
> > >
> > >
> > > On Tue, Feb 4, 2014 at 5:03 PM, Alejandro Abdelnur <[EMAIL PROTECTED]
> > >wrote:
> > >
> > >> IMO YARN-1577 is a blocker, it is breaking unmanaged AMs in a very odd
> > >> ways
> > >> (to the point it seems un-deterministic).
> > >>
> > >> I'd say eiher YARN-1577 is fixed or we revert
> > >> YARN-1493/YARN-1490/YARN-1166/YARN-1041/YARN-1566 (almost clean
> reverts)
> > >> from Hadoop 2.3 branch before doing the release.
> > >>
> > >>
> > >> I've verified that after reverting those JIRAs things work fine with
> > >> unmanaged AMs.
> > >>
> > >> Thanks.
> > >>
> > >>
> > >>
> > >>
> > >> On Tue, Feb 4, 2014 at 11:45 AM, Arun C Murthy <[EMAIL PROTECTED]>
> > >> wrote:
> > >>
> > >> > I punted YARN-1444 to 2.4 since it's a long-standing issue.
> > >> >
> > >> > Jian is away and I don't see YARN-1577 & YARN-1206 making much
> > progress
> > >> > till he is back; so I'm inclined to push both to 2.4 too. Any
> > >> objections?
> > >> >
> > >> > Looks like Daryn has both HADOOP-10301 & HDFS-4564 covered.
> > >> >
> > >> > Overall, I'll try get this out in next couple of days if we can
> clear
> > >> the
> > >> > list.
> > >> >
> > >> > thanks,
> > >> > Arun
> > >> >
> > >> > On Feb 3, 2014, at 12:14 PM, Arun C Murthy <[EMAIL PROTECTED]>
> > wrote:
> > >> >
> > >> > > An update. Per https://s.apache.org/hadoop-2.3.0-blockers we are
> > now
> > >> > down to 5 blockers: 1 Common, 1 HDFS, 3 YARN.
> > >> > >
> > >> > > Daryn (thanks!) has both the non-YARN covered. Vinod is helping
> out
> > >> with
> > >> > the YARN ones.
> > >> > >
> > >> > > thanks,
> > >> > > Arun
> > >> > >
> > >> > >
> > >> > >
> > >> >
> > >> > --
> > >> > Arun C. Murthy
> > >> > Hortonworks Inc.
> > >> > http://hortonworks.com/
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > CONFIDENTIALITY NOTICE
> > >> > NOTICE: This message is intended for the use of the individual or
> > >> entity to
> > >> > which it is addressed and may contain information that is
> > confidential,
> > >> > privileged and exempt from disclosure under applicable law. If the
> > >> reader
> > >> > of this message is not the intended recipient, you are hereby
> notified
> > >> that
> > >> > any printing, copying, dissemination, distribution, disclosure or
> > >> > forwarding of this communication is strictly prohibited. If you have
> > >> > received this communication in error, please contact the sender
> > >> immediately
> > >> > and delete it from your system. Thank You.
> > >
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB