Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> possibly Pig throttles the number of mappers


Copy link to this message
-
possibly Pig throttles the number of mappers
Hi,

We've seen a strange problem where some Pig jobs would just run fewer
mappers concurrently than the mapper capacity. Specifically we have a 10
node cluster and each is configured to have 12 mappers. Normally we have 120
mappers running. But for some Pig jobs it will only have 10 mappers running
(while nothing else is running), and actually appears to be 1 mapper per
node.

We have not noticed the same problem with other non-Pig hadoop job. Anyone
has experienced the same thing and have any explanation or remedy?

Thanks!
Dexin