Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> number of mapred slots


+
alxsss@... 2012-12-18, 06:08
Copy link to this message
-
Re: number of mapred slots
I think the rule of thumb (hortonworks at least) is 2x cores for maps
threads and 1x cores for reducers.  Don't have my notes here so I'm not
100%.  It's just a guideline in any event. :)

TEST, TEST, TEST.  :)

On Tue, Dec 18, 2012 at 1:08 AM, <[EMAIL PROTECTED]> wrote:

> Hello,
>
> I was unable to find any information regarding relationship between
> mapred slots and number of cpus on the net. All I found was that it is
> advisable to schedule two processes for one cpu.  If this is true, then for
> a slave  node with dual core( two cpus) that runs datanode, tasktracker,
> hbase regionserver and zookeeper, theoretically there is no space to run
> an additional mapred task. Any comment on this is welcome.
>
> In general what is the mapred slot and how is it related to number of cpu
> cores?
>
> Thanks in advance.
> Alex.
>
+
alxsss@... 2012-12-18, 06:39