Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> Pig 0.10 on YARN


+
Konstantin Boudnik 2013-03-29, 06:32
+
Dmitriy Ryaboy 2013-03-29, 16:08
+
Konstantin Boudnik 2013-03-29, 17:38
Copy link to this message
-
Re: Pig 0.10 on YARN
I missed the part where you are running it from JUnit. Still suspect a
classpath issue :). Can you dump your classpath as seen by JUnit, and what
happens when you run "-secretDebugCmd" and see if something is missing?

-D
On Fri, Mar 29, 2013 at 10:38 AM, Konstantin Boudnik <[EMAIL PROTECTED]> wrote:

> There's no problem with Pig itself. If I use Pig CLI I can run some
> PigLatin
> scripts and they go all the way to the completion.
>
> The problem arises while running TestPigTest _programmatically_ e.g. using
> JUnit runner. Is there any secret debug options in this case? I've looked
> through the code but didn't spot anything immediately.
>
> I will give "-secretDebugCmd" a try anyway - it might give me some ideas...
>
> Appreciate the help!
>   Cos
>
> On Fri, Mar 29, 2013 at 09:08AM, Dmitriy Ryaboy wrote:
> > Can you run `pig -secretDebugCmd` and look at your paths? I suspect
> there's
> > something going on there, like not having the right configs or hadoop
> jars.
> >
> >
> > On Thu, Mar 28, 2013 at 11:32 PM, Konstantin Boudnik <[EMAIL PROTECTED]>
> wrote:
> >
> > > Guys,
> > >
> > > I am trying to run TestPigTest against a fanctional Hadoop 2.0 YARN
> > > cluster -
> > > I can run MR jobs using Hive or plain MR examples. However, it is going
> > > somewhat unsuccessful. While using pig "far" jar file - e.g. with
> > > everything
> > > repacked inside of it - I seeminly can run TestPigTest against a real
> > > cluster.
> > >
> > > However, while using pig-withouthadoop.jar and adding all needed
> > > dependencies
> > > to the classpath, Pig is failing to communicate with YARN nor submit
> any MR
> > > jobs. There's no exceptions, not error messages anywhere in the logs.
> Have
> > > anyone seen anything similar?
> > >
> > > Any input is highly appreciated!
> > >
> > > --
> > > With regards,
> > >         Cos
> > >
> > >
>
+
Konstantin Boudnik 2013-03-29, 21:54
+
Johnny Zhang 2013-03-29, 22:25
+
Konstantin Boudnik 2013-03-29, 22:45
+
Konstantin Boudnik 2013-04-09, 06:41