I'm trying to prove that my cluster will in fact support multiple reducers,
the wordcount example doesn't seem to spawn more that one (1). Is that
correct? Is there a sure fire way to prove my cluster is configured
correctly in terms of launching the maximum (say two per node) number of
mappers and reducers?

Thanks!

You need to specify a non-default number of reducers, I've posted about this once before at: http://search-hadoop.com/m/gOpLh1yOq92

On 29-Nov-2011, at 8:03 PM, Hoot Thompson wrote:

> I'm trying to prove that my cluster will in fact support multiple reducers,
> the wordcount example doesn't seem to spawn more that one (1). Is that
> correct? Is there a sure fire way to prove my cluster is configured
> correctly in terms of launching the maximum (say two per node) number of
> mappers and reducers?
>
> Thanks!
>
>

Hi Hoot

You can specify the number of reducers explicitly using -D
mapred.reduce.tasks=n.

hadoop jar wordcount.jar com.wc.WordCount –D mapred.reduce.tasks=n /input
/output

Currenty your word count is triggering just 1 reducer because the defaukt
value of mapred.reduce.tasks woulld be set as 1 in your configuration file

Hope it helps !.

Regards
Bejoy.K.S

On Tue, Nov 29, 2011 at 8:03 PM, Hoot Thompson <[EMAIL PROTECTED]> wrote:

> I'm trying to prove that my cluster will in fact support multiple reducers,
> the wordcount example doesn't seem to spawn more that one (1). Is that
> correct? Is there a sure fire way to prove my cluster is configured
> correctly in terms of launching the maximum (say two per node) number of
> mappers and reducers?
>
> Thanks!
>
>
>

This works! Thank you very much!

-----Original Message-----
From: Bejoy Ks [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, November 29, 2011 9:45 AM
To: [EMAIL PROTECTED]
Subject: Re: Multiple reducers

Hi Hoot

You can specify the number of reducers explicitly using -D
mapred.reduce.tasks=n.

hadoop jar wordcount.jar com.wc.WordCount -D mapred.reduce.tasks=n /input
/output

Currenty your word count is triggering just 1 reducer because the defaukt
value of mapred.reduce.tasks woulld be set as 1 in your configuration file

Hope it helps !.

Regards
Bejoy.K.S

On Tue, Nov 29, 2011 at 8:03 PM, Hoot Thompson <[EMAIL PROTECTED]> wrote:

> I'm trying to prove that my cluster will in fact support multiple
reducers,
> the wordcount example doesn't seem to spawn more that one (1). Is that
> correct? Is there a sure fire way to prove my cluster is configured
> correctly in terms of launching the maximum (say two per node) number of
> mappers and reducers?
>
> Thanks!
>
>
>

Hi

Try setting the parameter mapred.reduce.tasks
hadoop jar hadoop-0.20.2-examples.jar  wordcount -D mapred.reduce.tasks=4 <Input> <Output>

Thanks.
Sumit

________________________________
 From: Hoot Thompson <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Tuesday, 29 November 2011 8:03 PM
Subject: Multiple reducers
 
I'm trying to prove that my cluster will in fact support multiple reducers,
the wordcount example doesn't seem to spawn more that one (1). Is that
correct? Is there a sure fire way to prove my cluster is configured
correctly in terms of launching the maximum (say two per node) number of
mappers and reducers?

Thanks!