Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Loading a custom schema

Siddhi Borkar 2013-09-26, 07:59
Prashant Kommireddi 2013-09-26, 08:16
Copy link to this message
RE: Loading a custom schema
Thanks Prashant, Will try this out.
-----Original Message-----
From: Prashant Kommireddi [mailto:[EMAIL PROTECTED]]
Sent: Thursday, September 26, 2013 1:47 PM
Subject: Re: Loading a custom schema

Hi Siddhi,

PigStorage by default looks for ".pig_schema" under the input dir. If you would like to use a different filename, you would have to override PigStorage.getSchema(String location, Job job) and define a custom JsonMetadata object. You might want to start here.

Using a schema file location completely outside of data files would involve passing the appropriate "schema path" location to JsonMetadata.getSchema.
On Thu, Sep 26, 2013 at 12:59 AM, Siddhi Borkar < [EMAIL PROTECTED]> wrote:

> Hi,
> I am trying to load a tsv file using PigStorage
> input_data = load 'input.tsv' using PigStorage('\t','-schema');
> This loads the tsv file as per the .pig_schema file present in the
> input folder.
> Is there any way to load the schema from a custom path? For ex, say I
> have the schema saved in a different directory and a different name
> (not.pig_schema) than the directory where input file is located.
> Thanks,
> Siddhi