Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Load Pig metadata from file?


Copy link to this message
-
Re: Load Pig metadata from file?
I think you need to play with some quotes, its more likely a bash problem.

one way to debug is bash -x pig  -f script.pig -param md=$(cat
metadata.dat) and check what does hadoop jar gets in the end.

try - md="$(cat metadata.dat)"
or -md="'$(cat metadata.dat)'" (single quote inside double quote
and so on..

Thanks,
Aniket

On Tue, May 15, 2012 at 3:34 PM, Saurabh S <[EMAIL PROTECTED]> wrote:

>
> Here is a sample LOAD statement from Programming Pig book:
>
> daily = load 'NYSE_daily' as (exchange:chararray, symbol:chararray,
>            date:chararray, open:float, high:float, low:float, close:float,
>            volume:int, adj_close:float);
>
> In my case, there are around 250 columns to load. So, I created a file,
> say, metadata.dat with its contents as follows:
>
>  (exchange:chararray, symbol:chararray,
>
>            date:chararray, open:float, high:float, low:float, close:float,
>
>            volume:int, adj_close:float)
>
> My load statement now looks like
>
> daily = load 'NYSE_daily' as $md;
>
> and the execution looks like.
>
> pig -f script.pig -param md=$(cat metadata.dat)
>
> However, I get the following error in this method:
>
> ERROR 1000: Error during parsing. Lexical error at line 9, column 0.
>  Encountered: <EOF> after : ""
>
> Copying the contents of the file in appropriate place works fine. But the
> pig script is cluttered with the metdata and I would like to separate it
> from the script. Any ideas?
>
> HCatLoader() does not seem to be available on my system.
>
>
>
>
--
"...:::Aniket:::... Quetzalco@tl"
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB