Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Local vs mapreduce mode


Copy link to this message
-
Local vs mapreduce mode

Dear Pig experts,

I have the following Pig script that works perfectly in local mode. However, in the mapreduce mode I get AU as :

$HADOOP_CONF_DIR fs -cat /scratch/AU/part-m-00000
Warning: $HADOOP_HOME is deprecated.

{}
{}
{}
{}

Both the local mode and the mapreduce mode relation A is set correctly.

Can anyone please tell me what are the recommended ways for debugging the script in mapreduce mode -- logging utilities etc.

REGISTER /users/p529444/software/pig-0.11.1/contrib/piggybank/java/piggybank.jar;
REGISTER /users/p529444/software/pig-0.11.1/parser.jar

DEFINE SequenceFileLoader org.apache.pig.piggybank.storage.SequenceFileLoader();

A = LOAD '/scratch/file.seq' USING SequenceFileLoader AS (key: chararray, value: chararray);
DESCRIBE A;
STORE A into '/scratch/A';

AU
 = FOREACH A GENERATE parser.Parser(key) AS {(id: int, class: chararray,
 name: chararray, begin: int, end: int, probone: chararray, probtwo:
chararray)};
STORE AU into '/scratch/AU';

     
+
Pradeep Gollakota 2013-11-05, 17:37
+
Sameer Tilak 2013-11-05, 17:46