Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> Unit test classpath trouble


Copy link to this message
-
Re: Unit test classpath trouble
Cheolsoo,

just to make sure I do not misinterpret your words:
 - there were a change in YARN packaging between 2.0.0 and 2.0.3 that move
   some APIs into separate artifact hadoop-yarn-client
 - adding hadoop-yarn-client is only relevant for 2.0.3 and later and won't
   work on 2.0.0-2.0.2 versions

Does that represent your point of view correctly? If so, increamenting the
version does't look like a big deal anyway, 'cause we are talking about
_minor_ Hadoop version increment, normally reflecting bug fixes, etc.

If the above mistates the situation that I don't see why the version needs to
be bumped up at all.

Please advise. With regards,
  Cos

On Mon, May 13, 2013 at 09:35AM, Cheolsoo Park wrote:
> Hi Cos,
>
> I am happy to backport it to branch-011. But I am not entirely sure whether
> we should bump the Hadoop version in a minor release. As of now,
> branch-0.11 unit tests run fine with 2.0.0-alpha.
>
> Please let me know if anyone has an opinion on this.
>
> Thanks,
> Cheolsoo
>
>
> On Sun, May 12, 2013 at 11:45 PM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:
>
> > If unit tests can't be run - which seems to be the case - it sound like a
> > release blocker to me. Any plans to back port this fix into 0.11.2?
> > 0.11.1 doesn't have it, right?
> >
> > Cos
> >
> > On Sun, May 12, 2013 at 09:12PM, Cheolsoo Park wrote:
> > > Hi Andrew,
> > >
> > > >> I'm trying HEAD of branch-0.11.
> > >
> > > You need PIG-3248: https://issues.apache.org/jira/browse/PIG-3248. It's
> > not
> > > in branch-0.11 but in trunk.
> > >
> > > In particular, you need to add the following lines to ivy.xml:
> > >
> > > +    <dependency org="org.apache.hadoop" name="hadoop-yarn-client"
> > > +      rev="${hadoop-mapreduce.version}" conf="hadoop23->master"/>
> > >
> > > Thanks,
> > > Cheolsoo
> > >
> > >
> > >
> > > On Sun, May 12, 2013 at 7:13 PM, Andrew Purtell <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > It happens consistently on multiple systems and different operating
> > systems
> > > > and JDK versions, so while is a build (or more specifically a unit test
> > > > environment) problem, it is unlikely an issue singular to me.
> > > >
> > > > On Saturday, May 11, 2013, Konstantin Boudnik wrote:
> > > >
> > > > > As you know we've been testing Pig 0.11 vs 2.0.4-alpha as a part of
> > > > > Bigtop's
> > > > > validation for the latest hadoop release and it worked ok. Bigtop
> > doesn't
> > > > > run
> > > > > unit tests though, so it seems like a build issue to me.
> > > > >
> > > > > Cos
> > > > >
> > > > > On Sat, May 11, 2013 at 09:19AM, Andrew Purtell wrote:
> > > > > > I've tried that, thanks. I did a bit more investigation and it
> > seems
> > > > the
> > > > > > issue is recent Hadoop 2 releases. Has anyone tried running Pig
> > unit
> > > > > tests
> > > > > > using a more recent Hadoop release than 2.0.0-alpha? Maybe my
> > trouble
> > > > is
> > > > > a
> > > > > > simple thing that someone with more experience with Pig internals
> > would
> > > > > see
> > > > > > right away? Cluster testing seems ok. It's just unit tests that
> > fail.
> > > > But
> > > > > > that is concerning.
> > > > > >
> > > > > > I'm trying HEAD of branch-0.11.
> > > > > >
> > > > > > My Java is version "1.6.0_43" Java(TM) SE Runtime Environment
> > (build
> > > > > > 1.6.0_43-b01) Java HotSpot(TM) 64-Bit Server VM (build 20.14-b01,
> > mixed
> > > > > > mode). OS is Ubuntu 13.04 (GNU/Linux 3.8.0-19-generic x86_64).
> > > > > >
> > > > > > With defaults and only -Dhadoopversion=23 on the Ant command line,
> > it
> > > > > seems
> > > > > > ok.
> > > > > >
> > > > > > With build.properties of:
> > > > > >
> > > > > > hadoopversion=23
> > > > > > hadoop-common.version=2.0.4-alpha
> > > > > > hadoop-hdfs.version=2.0.4-alpha
> > > > > > hadoop-mapreduce.version=2.0.4-alpha
> > > > > >
> > > > > >
> > > > > > or defined on the Ant command line, I'll see unit test failures
> > like:
> > > > > >
> > > > > > Testcase: testAccumWithDistinct took 0.868 sec
> > > > > >         Caused an ERROR
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB