Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # user - Num Reducers is Very Small!


+
唐亮 2011-09-15, 07:47
+
Alan Gates 2011-09-15, 14:44
+
jiang licht 2011-09-16, 01:31
+
jiang licht 2011-09-15, 17:16
Copy link to this message
-
Re: Num Reducers is Very Small!
Norbert Burger 2011-09-15, 23:35
On Thu, Sep 15, 2011 at 1:16 PM, jiang licht <[EMAIL PROTECTED]> wrote:
> Lower priority jobs then have to either wait for availability of reducer
> resources and idle, or get smaller number of slots than configured...

This last part ("or get smaller number...") isn't right, is it?  The number
of reducer tasks is a function of the Pig script design.  While any
particular job can be forced to wait until it receives enough slots to
complete processing, the cluster state shouldn't force a change in script
configuration, right?

Norbert
+
jiang licht 2011-09-16, 01:24
+
Thejas Nair 2011-09-16, 00:59
+
jiang licht 2011-09-16, 01:29