Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> How can I know how many mappers created?


+
谭军 2011-08-07, 06:45
Copy link to this message
-
RE: How can I know how many mappers created?
Hi Jun,

 

Request you to go through this link:

http://wiki.apache.org/hadoop/HowManyMapsAndReduces

 

Following configuration may help you :

mapred.min.split.size

mapred.map.tasks (This is just a hint to InputFormat)

 

But how can I know how many mappers have been created while doing my job?

The implementation of InputFormat actually defines how many Mappers are
going to be in the submitted job.

 

Can I do some sets to make it only 2 mappers created and each mapper runs on
one node?

The scheduler implementation has the logic to schedule mappers and reducers.

 

 

 

Regards,
Subroto Sanyal

  _____  

From: 谭军 [mailto:[EMAIL PROTECTED]]
Sent: Sunday, August 07, 2011 12:15 PM
To: [EMAIL PROTECTED]
Subject: How can I know how many mappers created?

 

Hi,

I just want 2 mappers created to do my job for there are only 2 data nodes.

I think that is the most efficient.

But how can I know how many mappers have been created while doing my job?

Can I do some sets to make it only 2 mappers created and each mapper runs on
one node?

Thanks

--

Best regards!

Jun Tan
+
tanjun_2525 2011-08-08, 05:25
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB