Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - access schema defined in LOAD statement in custom LoadFunc?

Copy link to this message
Re: access schema defined in LOAD statement in custom LoadFunc?
Dmitriy Ryaboy 2012-09-17, 05:06
I am not sure why pushProjection doesn't solve your dilemma?
This is what we use in HBaseStorage, and ElephantBird uses in thrift
and protobuf loaders.


On Sun, Sep 16, 2012 at 8:11 PM, Jim Donofrio <[EMAIL PROTECTED]> wrote:
> I guess a workaround could be to Base64 decode the pig.script property and
> look for AS statements
> On 09/15/2012 12:38 PM, Alan Gates wrote:
>> Unfortunately, no.  I agree we should add that to the LoadFunc interface.
>> Alan.
>> On Sep 15, 2012, at 1:13 AM, Jim Donofrio wrote:
>>> Is there anyway within a LoadFunc to access the schema that a user
>>> defines after AS in a LOAD statement? Is there some property I can access in
>>> the UDFContext or ? pushProjection provides the schema from a FOREACH
>>> GENERATE and getSchema seems to be only meant to read part of the actual
>>> file or a JSON schema file off disk.