Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - Num Reducers is Very Small!


Copy link to this message
-
Re: Num Reducers is Very Small!
Norbert Burger 2011-09-15, 23:35
On Thu, Sep 15, 2011 at 1:16 PM, jiang licht <[EMAIL PROTECTED]> wrote:
> Lower priority jobs then have to either wait for availability of reducer
> resources and idle, or get smaller number of slots than configured...

This last part ("or get smaller number...") isn't right, is it?  The number
of reducer tasks is a function of the Pig script design.  While any
particular job can be forced to wait until it receives enough slots to
complete processing, the cluster state shouldn't force a change in script
configuration, right?

Norbert