Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - different mapred.min.split.size within one pig script?


Copy link to this message
-
different mapred.min.split.size within one pig script?
Yang 2012-06-11, 02:06
I need to set mapred.min.split.size for one part of my pig script
because the mapper job corresponding to the first part of the script takes
much longer time per input record than other parts of the script.

so I have to set the split size very small to take care of that particular
script,

but then later parts of the script also used this value and used too many
splits,

is it possible to set min.split.size value to different values within the
same script?

Thanks
Yang