Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> More on issue with local vs mapreduce mode


Copy link to this message
-
Re: More on issue with local vs mapreduce mode
"The same script does not work in the mapreduce mode. "
What does it mean?
2013/11/6 Sameer Tilak <[EMAIL PROTECTED]>

> Hello,
>
> My script in the local mode works perfectly. The same script does not work
> in the mapreduce mode. For the local mode, the o/p is saved in the current
> directory, where as for the mapreduce mode I use /scrach directory on HDFS.
>
> Local mode:
>
> A = LOAD 'file.seq' USING SequenceFileLoader AS (key: chararray, value:
> chararray);
> DESCRIBE A;
> STORE A into 'A';
>
> AU = FOREACH A GENERATE FLATTEN(parser.customFilter(key,'AAAAA'));
> STORE AU into 'AU';
>
>
> Mapreduce mode:
>
> A = LOAD '/scratch/file.seq' USING SequenceFileLoader AS (key: chararray,
> value: chararray);
> DESCRIBE A;
> STORE A into '/scratch/A';
>
> AU = FOREACH A GENERATE FLATTEN(parser.customFilter(key,'AAAAA'));
> STORE AU into '/scratch/AU';
>
> Can someone please point me to tools that I can use to debug the script in
> mapreduce mode? Also, any thoughts on why this might be happening would be
> great!
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB