Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Num Reducers is Very Small!


+
唐亮 2011-09-15, 07:47
+
Alan Gates 2011-09-15, 14:44
+
jiang licht 2011-09-16, 01:31
+
jiang licht 2011-09-15, 17:16
Copy link to this message
-
Re: Num Reducers is Very Small!
On Thu, Sep 15, 2011 at 1:16 PM, jiang licht <[EMAIL PROTECTED]> wrote:
> Lower priority jobs then have to either wait for availability of reducer
> resources and idle, or get smaller number of slots than configured...

This last part ("or get smaller number...") isn't right, is it?  The number
of reducer tasks is a function of the Pig script design.  While any
particular job can be forced to wait until it receives enough slots to
complete processing, the cluster state shouldn't force a change in script
configuration, right?

Norbert
+
jiang licht 2011-09-16, 01:24
+
Thejas Nair 2011-09-16, 00:59
+
jiang licht 2011-09-16, 01:29
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB