Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Setting number of mappers in Teragen


+
anil gupta 2012-12-26, 18:19
Copy link to this message
-
Re: Setting number of mappers in Teragen
The MR1 teragen's mappers # depends on the total number of rows and
demanded # of maps.

How are you passing -Dmapred.map.tasks=20 (no spaces) exactly? All
generic options must go in before any other options do, so it should
appear right after the word "teragen" in your command.

On Wed, Dec 26, 2012 at 11:49 PM, anil gupta <[EMAIL PROTECTED]> wrote:
> Hi All,
>
> I have 5 worker nodes and i have 4 map slots per node. So, i have 20 map
> slots in my cluster. But when, i start my Teragen job, it only spawns 2
> mappers for entire job. I have even tried using the option
> -Dmapred.map.tasks = 20 . Can anyone tell me how to force teragen to use 20
> mappers for generating the data? I am using cdh4.1.2 with Mapreducev1(Hadoop
> 0.20.2)
> --
> Thanks & Regards,
> Anil Gupta

--
Harsh J
+
anil gupta 2012-12-26, 18:41
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB