Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Local vs mapreduce mode


Copy link to this message
-
Re: Local vs mapreduce mode
Really dumb question but... when running in MapReduce mode, is your input
file on HDFS?
On Tue, Nov 5, 2013 at 9:17 AM, Sameer Tilak <[EMAIL PROTECTED]> wrote:

>
> Dear Pig experts,
>
> I have the following Pig script that works perfectly in local mode.
> However, in the mapreduce mode I get AU as :
>
> $HADOOP_CONF_DIR fs -cat /scratch/AU/part-m-00000
> Warning: $HADOOP_HOME is deprecated.
>
> {}
> {}
> {}
> {}
>
> Both the local mode and the mapreduce mode relation A is set correctly.
>
> Can anyone please tell me what are the recommended ways for debugging the
> script in mapreduce mode -- logging utilities etc.
>
> REGISTER
> /users/p529444/software/pig-0.11.1/contrib/piggybank/java/piggybank.jar;
> REGISTER /users/p529444/software/pig-0.11.1/parser.jar
>
> DEFINE SequenceFileLoader
> org.apache.pig.piggybank.storage.SequenceFileLoader();
>
> A = LOAD '/scratch/file.seq' USING SequenceFileLoader AS (key: chararray,
> value: chararray);
> DESCRIBE A;
> STORE A into '/scratch/A';
>
> AU
>  = FOREACH A GENERATE parser.Parser(key) AS {(id: int, class: chararray,
>  name: chararray, begin: int, end: int, probone: chararray, probtwo:
> chararray)};
> STORE AU into '/scratch/AU';
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB