Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Spindle per Cores


Copy link to this message
-
Re: Spindle per Cores
Thanks Ted,

Sorry I was too rush to type. I was asking should I pick number 8 (not count the Hyperthread)
or number 16 (count the Hyperthreads) in order to adjust spindle and cores ratios
From: Ted Dunning <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Fri, 12 Oct 2012 12:11:56 -0700
To: <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Spindle per Cores

It depends on your distribution.  Some distributions are more efficient at driving spindles than others.

Ratios as high as 2 spindles per core are sometimes quite reasonable.

On Fri, Oct 12, 2012 at 10:46 AM, Patai Sangbutsarakum <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
I have read around about the hardware recommendation for hadoop cluster.
One of them is recommend 1:1 ratio between spindle per core.

Intel CPU come with Hyperthread which will double the number cores on
one physical CPU. eg. 8 cores with Hyperthread it because 16 which is
where we start to calculate about number of task slots per node.

Once it come to spindle, i strongly believe I should pick 8 cores and
picks 8 disks in order to get 1:1 ratio.

Please suggest
Patai

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB