Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> How to configure multiple reduce jobs in hadoop 2.2.0


Copy link to this message
-
Re: How to configure multiple reduce jobs in hadoop 2.2.0
Also, you should be limited by your container configuration at
yarn-site.xml and mapred-site.xml, check
THIS<http://hortonworks.com/blog/how-to-plan-and-configure-yarn-in-hdp-2-0/>to
understand how resource management works.

Basically you can set the number of reducers you want but you are limited
to the number the system can actually hold by the configuration you have
set.

Hope it helps.
Regards,
Silvina
On 16 January 2014 08:54, sudhakara st <[EMAIL PROTECTED]> wrote:

> Hello Ashish,
>
> Using “-D mapreduce.job.reduces=number” with fixed number of reducer will
> spawn that many for a job.
>
>
> On Thu, Jan 16, 2014 at 12:45 PM, Ashish Jain <[EMAIL PROTECTED]> wrote:
>
>> Dear All,
>>
>> I have a 3 node cluster and have a map reduce job running on it. I have 8
>> data blocks spread across all the 3 nodes. While running map reduce job I
>> could see 8 map tasks running however reduce job is only 1. Is there a way
>> to configure multiple reduce jobs?
>>
>> --Ashish
>>
>
>
>
> --
>
> Regards,
> ...Sudhakara.st
>
>