Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> who is resetting my mapred.map.tasks?


Copy link to this message
-
who is resetting my mapred.map.tasks?
I have the following PIG script.
In the beginning, I set the mapred.map.tasks
but when the job is launched,  I see from jobtracker UI that the job.xml
shows that
the mapred.map.tasks param is set to "50". but I never used such a value.
what is resetting this value ?

this is pig-0.10.0

Thanks
Yang
################################

SET mapred.min.split.size  10000;
SET pig.noSplitCombination true;

-- SET mapred.map.tasks.speculative.execution false;
-- SET mapred.reduce.tasks.speculative.execution false;
SET mapred.map.tasks 1020;
set default_parallel 42;

verdict = LOAD '$input' AS ( partition_key:chararray);

xx = FILTER verdict by partition_key == '';

dump xx;
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB