Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> possibly Pig throttles the number of mappers

Copy link to this message
possibly Pig throttles the number of mappers

We've seen a strange problem where some Pig jobs would just run fewer
mappers concurrently than the mapper capacity. Specifically we have a 10
node cluster and each is configured to have 12 mappers. Normally we have 120
mappers running. But for some Pig jobs it will only have 10 mappers running
(while nothing else is running), and actually appears to be 1 mapper per

We have not noticed the same problem with other non-Pig hadoop job. Anyone
has experienced the same thing and have any explanation or remedy?