Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Unoverride load in PigUint


Copy link to this message
-
Re: Unoverride load in PigUint
Ruslan Al-Fakikh 2013-12-21, 17:05
It seems to be a heavy PigUnit limitation. Maybe you can open a jira for
this?:)

Thanks,
Ruslan
On Sat, Dec 21, 2013 at 2:01 AM, Serega Sheypak <[EMAIL PROTECTED]>wrote:

> No :)
> I did extend PigUnit to work with several input relations (i mean 1..N load
> statements), but I didn't find a  way to use "native" loader/storage.
> Looks like that the only solution is to create wrapper: data-driven tester
> which feeds script to local pig server and verifies output.
> We did in Megafon. I tried to use "recommended approach" - pig unit for my
> own purposes.
> Fail.
>
>
> 2013/12/21 Ruslan Al-Fakikh <[EMAIL PROTECTED]>
>
> > Hi Serega!
> >
> > Have you resolved the issue? I am going to encounter the same problem,
> but
> > I don't know a solution.
> >
> > Thanks
> >
> >
> > On Sun, Dec 15, 2013 at 6:07 PM, Serega Sheypak <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi!
> > > By default PigUnit does override LOAD statements
> > > Is there any possiblity to void this?
> > > I'm using AvroStorage because of evolving schemas and I would like to
> > write
> > > in my tests:
> > >
> > > --load test dataset
> > > in = LOAD '$pathToIn' using AvroStorage();
> > >
> > > --project the fields I need
> > > inProjected = FOREACH in GENERATE f1, f2;
> > >
> > > Right now I can't do this. PigUnit requires explicit field projection
> on
> > > LOAD statement.
> > >
> >
>