Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> Num Reducers is Very Small!


Copy link to this message
-
Re: Num Reducers is Very Small!
On Thu, Sep 15, 2011 at 1:16 PM, jiang licht <[EMAIL PROTECTED]> wrote:
> Lower priority jobs then have to either wait for availability of reducer
> resources and idle, or get smaller number of slots than configured...

This last part ("or get smaller number...") isn't right, is it?  The number
of reducer tasks is a function of the Pig script design.  While any
particular job can be forced to wait until it receives enough slots to
complete processing, the cluster state shouldn't force a change in script
configuration, right?

Norbert
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB