Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Unoverride load in PigUint


Copy link to this message
-
Re: Unoverride load in PigUint
I did
On Sat, Dec 21, 2013 at 9:44 PM, Serega Sheypak <[EMAIL PROTECTED]>wrote:

> https://issues.apache.org/jira/browse/PIG-3638
> "like" it :)
>
>
> 2013/12/21 Ruslan Al-Fakikh <[EMAIL PROTECTED]>
>
> > It seems to be a heavy PigUnit limitation. Maybe you can open a jira for
> > this?:)
> >
> > Thanks,
> > Ruslan
> >
> >
> > On Sat, Dec 21, 2013 at 2:01 AM, Serega Sheypak <
> [EMAIL PROTECTED]
> > >wrote:
> >
> > > No :)
> > > I did extend PigUnit to work with several input relations (i mean 1..N
> > load
> > > statements), but I didn't find a  way to use "native" loader/storage.
> > > Looks like that the only solution is to create wrapper: data-driven
> > tester
> > > which feeds script to local pig server and verifies output.
> > > We did in Megafon. I tried to use "recommended approach" - pig unit for
> > my
> > > own purposes.
> > > Fail.
> > >
> > >
> > > 2013/12/21 Ruslan Al-Fakikh <[EMAIL PROTECTED]>
> > >
> > > > Hi Serega!
> > > >
> > > > Have you resolved the issue? I am going to encounter the same
> problem,
> > > but
> > > > I don't know a solution.
> > > >
> > > > Thanks
> > > >
> > > >
> > > > On Sun, Dec 15, 2013 at 6:07 PM, Serega Sheypak <
> > > [EMAIL PROTECTED]
> > > > >wrote:
> > > >
> > > > > Hi!
> > > > > By default PigUnit does override LOAD statements
> > > > > Is there any possiblity to void this?
> > > > > I'm using AvroStorage because of evolving schemas and I would like
> to
> > > > write
> > > > > in my tests:
> > > > >
> > > > > --load test dataset
> > > > > in = LOAD '$pathToIn' using AvroStorage();
> > > > >
> > > > > --project the fields I need
> > > > > inProjected = FOREACH in GENERATE f1, f2;
> > > > >
> > > > > Right now I can't do this. PigUnit requires explicit field
> projection
> > > on
> > > > > LOAD statement.
> > > > >
> > > >
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB