Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Use HDFS file as input when running pig with local mode


Copy link to this message
-
Re: Use HDFS file as input when running pig with local mode
Running Pig in localmode means to use local hdfs and m/r framework. There
is no separate concept of localmode for pig apart from hdfs and m/r
framework on which it runs.

Regards,
Shahab
On Fri, Oct 11, 2013 at 10:36 PM, Gordon Wang <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Pig will not load any hadoop conf when running with local mode.
> Does this means that I can not use HDFS files as input when I run pig with
> local mode?
>
> Say, I have the following script
> ================================> mytable = LOAD 'hdfs://namenodehost/user/hadoop/test.csv'
>           USING PigStorage(',')
>           AS (col1:chararray, col2:chararray, col3:int);
> dump mytable;
> =================================>
> When I run the script with local mode, it can not load files. So I want to
> debug my script, and I have to change the script to read data from local
> FS.
>
> Do you have any best practice to debug pig script without changing the
> input path?
>
> Thanks.
>
> --
> Regards
> Gordon Wang
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB