Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Loading a custom schema


Copy link to this message
-
RE: Loading a custom schema
Thanks Prashant, Will try this out.
-----Original Message-----
From: Prashant Kommireddi [mailto:[EMAIL PROTECTED]]
Sent: Thursday, September 26, 2013 1:47 PM
To: [EMAIL PROTECTED]
Subject: Re: Loading a custom schema

Hi Siddhi,

PigStorage by default looks for ".pig_schema" under the input dir. If you would like to use a different filename, you would have to override PigStorage.getSchema(String location, Job job) and define a custom JsonMetadata object. You might want to start here.

Using a schema file location completely outside of data files would involve passing the appropriate "schema path" location to JsonMetadata.getSchema.
On Thu, Sep 26, 2013 at 12:59 AM, Siddhi Borkar < [EMAIL PROTECTED]> wrote:

> Hi,
>
> I am trying to load a tsv file using PigStorage
>
> input_data = load 'input.tsv' using PigStorage('\t','-schema');
>
> This loads the tsv file as per the .pig_schema file present in the
> input folder.
>
> Is there any way to load the schema from a custom path? For ex, say I
> have the schema saved in a different directory and a different name
> (not.pig_schema) than the directory where input file is located.
>
> Thanks,
> Siddhi
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB