Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Unoverride load in PigUint


+
Serega Sheypak 2013-12-15, 14:07
+
Ruslan Al-Fakikh 2013-12-20, 21:53
+
Serega Sheypak 2013-12-20, 22:01
+
Ruslan Al-Fakikh 2013-12-21, 17:05
+
Serega Sheypak 2013-12-21, 17:44
Copy link to this message
-
Re: Unoverride load in PigUint
I did
On Sat, Dec 21, 2013 at 9:44 PM, Serega Sheypak <[EMAIL PROTECTED]>wrote:

> https://issues.apache.org/jira/browse/PIG-3638
> "like" it :)
>
>
> 2013/12/21 Ruslan Al-Fakikh <[EMAIL PROTECTED]>
>
> > It seems to be a heavy PigUnit limitation. Maybe you can open a jira for
> > this?:)
> >
> > Thanks,
> > Ruslan
> >
> >
> > On Sat, Dec 21, 2013 at 2:01 AM, Serega Sheypak <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > No :)
> > > I did extend PigUnit to work with several input relations (i mean 1..N
> > load
> > > statements), but I didn't find a  way to use "native" loader/storage.
> > > Looks like that the only solution is to create wrapper: data-driven
> > tester
> > > which feeds script to local pig server and verifies output.
> > > We did in Megafon. I tried to use "recommended approach" - pig unit for
> > my
> > > own purposes.
> > > Fail.
> > >
> > >
> > > 2013/12/21 Ruslan Al-Fakikh <[EMAIL PROTECTED]>
> > >
> > > > Hi Serega!
> > > >
> > > > Have you resolved the issue? I am going to encounter the same
> problem,
> > > but
> > > > I don't know a solution.
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Sun, Dec 15, 2013 at 6:07 PM, Serega Sheypak <
> > > [EMAIL PROTECTED]
> > > > >wrote:
> > > >
> > > > > Hi!
> > > > > By default PigUnit does override LOAD statements
> > > > > Is there any possiblity to void this?
> > > > > I'm using AvroStorage because of evolving schemas and I would like
> to
> > > > write
> > > > > in my tests:
> > > > >
> > > > > --load test dataset
> > > > > in = LOAD '$pathToIn' using AvroStorage();
> > > > >
> > > > > --project the fields I need
> > > > > inProjected = FOREACH in GENERATE f1, f2;
> > > > >
> > > > > Right now I can't do this. PigUnit requires explicit field
> projection
> > > on
> > > > > LOAD statement.
> > > > >
> > > >
> > >
> >
>