Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Num Reducers is Very Small!

唐亮 2011-09-15, 07:47
Alan Gates 2011-09-15, 14:44
jiang licht 2011-09-16, 01:31
jiang licht 2011-09-15, 17:16
Copy link to this message
Re: Num Reducers is Very Small!
On Thu, Sep 15, 2011 at 1:16 PM, jiang licht <[EMAIL PROTECTED]> wrote:
> Lower priority jobs then have to either wait for availability of reducer
> resources and idle, or get smaller number of slots than configured...

This last part ("or get smaller number...") isn't right, is it?  The number
of reducer tasks is a function of the Pig script design.  While any
particular job can be forced to wait until it receives enough slots to
complete processing, the cluster state shouldn't force a change in script
configuration, right?

jiang licht 2011-09-16, 01:24
Thejas Nair 2011-09-16, 00:59
jiang licht 2011-09-16, 01:29