Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> More on issue with local vs mapreduce mode


Copy link to this message
-
Re: More on issue with local vs mapreduce mode
"The same script does not work in the mapreduce mode. "
What does it mean?
2013/11/6 Sameer Tilak <[EMAIL PROTECTED]>

> Hello,
>
> My script in the local mode works perfectly. The same script does not work
> in the mapreduce mode. For the local mode, the o/p is saved in the current
> directory, where as for the mapreduce mode I use /scrach directory on HDFS.
>
> Local mode:
>
> A = LOAD 'file.seq' USING SequenceFileLoader AS (key: chararray, value:
> chararray);
> DESCRIBE A;
> STORE A into 'A';
>
> AU = FOREACH A GENERATE FLATTEN(parser.customFilter(key,'AAAAA'));
> STORE AU into 'AU';
>
>
> Mapreduce mode:
>
> A = LOAD '/scratch/file.seq' USING SequenceFileLoader AS (key: chararray,
> value: chararray);
> DESCRIBE A;
> STORE A into '/scratch/A';
>
> AU = FOREACH A GENERATE FLATTEN(parser.customFilter(key,'AAAAA'));
> STORE AU into '/scratch/AU';
>
> Can someone please point me to tools that I can use to debug the script in
> mapreduce mode? Also, any thoughts on why this might be happening would be
> great!
>
>