Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Pig 0.10 on YARN


+
Konstantin Boudnik 2013-03-29, 06:32
+
Dmitriy Ryaboy 2013-03-29, 16:08
+
Konstantin Boudnik 2013-03-29, 17:38
Copy link to this message
-
Re: Pig 0.10 on YARN
I missed the part where you are running it from JUnit. Still suspect a
classpath issue :). Can you dump your classpath as seen by JUnit, and what
happens when you run "-secretDebugCmd" and see if something is missing?

-D
On Fri, Mar 29, 2013 at 10:38 AM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:

> There's no problem with Pig itself. If I use Pig CLI I can run some
> PigLatin
> scripts and they go all the way to the completion.
>
> The problem arises while running TestPigTest _programmatically_ e.g. using
> JUnit runner. Is there any secret debug options in this case? I've looked
> through the code but didn't spot anything immediately.
>
> I will give "-secretDebugCmd" a try anyway - it might give me some ideas...
>
> Appreciate the help!
>   Cos
>
> On Fri, Mar 29, 2013 at 09:08AM, Dmitriy Ryaboy wrote:
> > Can you run `pig -secretDebugCmd` and look at your paths? I suspect
> there's
> > something going on there, like not having the right configs or hadoop
> jars.
> >
> >
> > On Thu, Mar 28, 2013 at 11:32 PM, Konstantin Boudnik <[EMAIL PROTECTED]>
> wrote:
> >
> > > Guys,
> > >
> > > I am trying to run TestPigTest against a fanctional Hadoop 2.0 YARN
> > > cluster -
> > > I can run MR jobs using Hive or plain MR examples. However, it is going
> > > somewhat unsuccessful. While using pig "far" jar file - e.g. with
> > > everything
> > > repacked inside of it - I seeminly can run TestPigTest against a real
> > > cluster.
> > >
> > > However, while using pig-withouthadoop.jar and adding all needed
> > > dependencies
> > > to the classpath, Pig is failing to communicate with YARN nor submit
> any MR
> > > jobs. There's no exceptions, not error messages anywhere in the logs.
> Have
> > > anyone seen anything similar?
> > >
> > > Any input is highly appreciated!
> > >
> > > --
> > > With regards,
> > >         Cos
> > >
> > >
>
+
Konstantin Boudnik 2013-03-29, 21:54
+
Johnny Zhang 2013-03-29, 22:25
+
Konstantin Boudnik 2013-03-29, 22:45
+
Konstantin Boudnik 2013-04-09, 06:41
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB