Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Loading a custom schema


+
Siddhi Borkar 2013-09-26, 07:59
Copy link to this message
-
Re: Loading a custom schema
Hi Siddhi,

PigStorage by default looks for ".pig_schema" under the input dir. If you
would like to use a different filename, you would have to override
PigStorage.getSchema(String location, Job job) and define a custom
JsonMetadata object. You might want to start here.

Using a schema file location completely outside of data files would involve
passing the appropriate "schema path" location to JsonMetadata.getSchema.
On Thu, Sep 26, 2013 at 12:59 AM, Siddhi Borkar <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am trying to load a tsv file using PigStorage
>
> input_data = load 'input.tsv' using PigStorage('\t','-schema');
>
> This loads the tsv file as per the .pig_schema file present in the input
> folder.
>
> Is there any way to load the schema from a custom path? For ex, say I have
> the schema saved in a different directory and a different name
> (not.pig_schema) than the directory where input file is located.
>
> Thanks,
> Siddhi
>
+
Siddhi Borkar 2013-10-03, 03:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB