Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Map Reduce slot


Copy link to this message
-
RE: Map Reduce slot
I checked my MR admin page running on <host>:50030:jobtracker.jsp this is what I am learning.

I run 2-core processor, so the admin page told me   that my max map/reduce slot capacity was  14 or so I assume 7 nodes x 2slots.

I did not touch the property of .map.tasks. It seemed that MR set it nicely to 14 automatically, processing 45 of my maps.  However, I increased reduce task from the default 1 to 14 in the mapred-site.xml of the JobTracker.

<property>
    <name>mapred.reduce.tasks</name>
    <value>14</value>
  </property>

The job worked really fast.

Running Map TasksRunning Reduce TasksTotal SubmissionsNodesOccupied Map SlotsOccupied Reduce SlotsReserved Map SlotsReserved Reduce SlotsMap Task CapacityReduce Task CapacityAvg. Tasks/NodeBlacklisted NodesExcluded Nodes14017<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=active>1400014144.000<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=blacklisted>1<http://ec2-54-242-185-163.compute-1.amazonaws.com:50030/machines.jsp?type=excluded>
From: kapil bhosale [mailto:[EMAIL PROTECTED]]
Sent: Thursday, November 01, 2012 3:51 AM
To: [EMAIL PROTECTED]
Subject: Re: Map Reduce slot

Hi Uddipan,

In my Opinion
 Hadoop Jobtracker Interface provides every Details Including
   # number of Map slots
   # number of Reducer slots

 and other Infromation can also be found in Log files generated.
 Which can also be explored through the same Interface.
 Jobtracker address : Masternode:your_port/jobtracker.jsp<http://>

Other useful details can also be found.

-
kapil

On Thu, Nov 1, 2012 at 1:11 PM, Uddipan Mukherjee <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Hi Hadoop Gurus.

  What is the good way to know following information in my Hadoop cluster.

# number of Map slots
# number of Reducer slots
JVM Heapsize for Map task
JVM heapsize for reduce task
reuse JVM flag

Thanks and Regards
Uddipan

**************** CAUTION - Disclaimer *****************
This e-mail contains PRIVILEGED AND CONFIDENTIAL INFORMATION intended solely
for the use of the addressee(s). If you are not the intended recipient, please
notify the sender by e-mail and delete the original message. Further, you are not
to copy, disclose, or distribute this e-mail or its contents to any other person and
any such actions are unlawful. This e-mail may contain viruses. Infosys has taken
every reasonable precaution to minimize this risk, but is not liable for any damage
you may sustain as a result of any virus in this e-mail. You should carry out your
own virus checks before opening the e-mail or attachment. Infosys reserves the
right to monitor and review the content of all messages sent to or from this e-mail
address. Messages sent to or from this e-mail address may be stored on the
Infosys e-mail system.
***INFOSYS******** End of Disclaimer ********INFOSYS***

--
kap's
NOTICE: This e-mail message and any attachments are confidential, subject to copyright and may be privileged. Any unauthorized use, copying or disclosure is prohibited. If you are not the intended recipient, please delete and contact the sender immediately. Please consider the environment before printing this e-mail. AVIS : le pr?sent courriel et toute pi?ce jointe qui l'accompagne sont confidentiels, prot?g?s par le droit d'auteur et peuvent ?tre couverts par le secret professionnel. Toute utilisation, copie ou divulgation non autoris?e est interdite. Si vous n'?tes pas le destinataire pr?vu de ce courriel, supprimez-le et contactez imm?diatement l'exp?diteur. Veuillez penser ? l'environnement avant d'imprimer le pr?sent courriel