Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> parallism level


Copy link to this message
-
Re: parallism level
Please see http://hadoop.apache.org/pig/docs/r0.7.0/piglatin_ref2.html .

You can use Oset default_parallel 10¹ to ask query to use 10 reducers for
all MR jobs, or specify Oparallel x¹ in the pig statement to ask pig to use
x number of reducers for that operation (for operations like group, order-by
, join that usually result in a separate MR job).
-Thejas

On 7/28/10 7:00 PM, "Gang Luo" <[EMAIL PROTECTED]> wrote:

> Hi all,
> by default the parallism (number of reducers) of a pig query is 1. How to
> change
> this value? If I set the value to 10, does that mean all the MR jobs for this
> query will run with 10 reducers?
>
>
> Thanks,
> -Gang
>
>
>      
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB