Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> access schema defined in LOAD statement in custom LoadFunc?


Copy link to this message
-
Re: access schema defined in LOAD statement in custom LoadFunc?
I am not sure why pushProjection doesn't solve your dilemma?
This is what we use in HBaseStorage, and ElephantBird uses in thrift
and protobuf loaders.

D

On Sun, Sep 16, 2012 at 8:11 PM, Jim Donofrio <[EMAIL PROTECTED]> wrote:
> I guess a workaround could be to Base64 decode the pig.script property and
> look for AS statements
>
>
> On 09/15/2012 12:38 PM, Alan Gates wrote:
>>
>> Unfortunately, no.  I agree we should add that to the LoadFunc interface.
>>
>> Alan.
>>
>> On Sep 15, 2012, at 1:13 AM, Jim Donofrio wrote:
>>
>>> Is there anyway within a LoadFunc to access the schema that a user
>>> defines after AS in a LOAD statement? Is there some property I can access in
>>> the UDFContext or ? pushProjection provides the schema from a FOREACH
>>> GENERATE and getSchema seems to be only meant to read part of the actual
>>> file or a JSON schema file off disk.
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB