Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> number of mapred slots


Copy link to this message
-
Re: number of mapred slots
I think the rule of thumb (hortonworks at least) is 2x cores for maps
threads and 1x cores for reducers.  Don't have my notes here so I'm not
100%.  It's just a guideline in any event. :)

TEST, TEST, TEST.  :)

On Tue, Dec 18, 2012 at 1:08 AM, <[EMAIL PROTECTED]> wrote:

> Hello,
>
> I was unable to find any information regarding relationship between
> mapred slots and number of cpus on the net. All I found was that it is
> advisable to schedule two processes for one cpu.  If this is true, then for
> a slave  node with dual core( two cpus) that runs datanode, tasktracker,
> hbase regionserver and zookeeper, theoretically there is no space to run
> an additional mapred task. Any comment on this is welcome.
>
> In general what is the mapred slot and how is it related to number of cpu
> cores?
>
> Thanks in advance.
> Alex.
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB