Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Which hardware to choose


Copy link to this message
-
Re: Which hardware to choose
Which is a reasonable number in this hardware?

On 10/02/2012 09:40 PM, Michael Segel wrote:
> I think he's saying that its 24 maps 8 reducers per node and at 48GB that could be too many mappers.
> Especially if they want to run HBase.
>
> On Oct 2, 2012, at 8:14 PM, hadoopman <[EMAIL PROTECTED]> wrote:
>
>> Only 24 map and 8 reduce tasks for 38 data nodes?  are you sure that's right?  Sounds VERY low for a cluster that size.
>>
>> We have only 10 c2100's and are running I believe 140 map and 70 reduce slots so far with pretty decent performance.
>>
>>
>>
>> On 10/02/2012 12:55 PM, Alexander Pivovarov wrote:
>>> 38 data nodes + 2 Name Nodes
>>>>   >
>>>>   >  Data Node:
>>>>   >  Dell PowerEdge C2100 series
>>>>   >  2 x XEON x5670
>>>>   >  48 GB RAM ECC  (12x4GB 1333MHz)
>>>>   >  12 x 2 TB  7200 RPM SATA HDD (with hot swap)  JBOD
>>>>   >  Intel Gigabit ET Dual port PCIe x4
>>>>   >  Redundant Power Supply
>>>>   >  Hadoop CDH3
>>>>   >  max map tasks 24
>>>>   >  max reduce tasks 8
>>
>
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci

--

Marcos Luis Ort�z Valmaseda
*Data Engineer && Sr. System Administrator at UCI*
about.me/marcosortiz <http://about.me/marcosortiz>
My Blog <http://marcosluis2186.posterous.com>
Tumblr's blog <http://marcosortiz.tumblr.com/>
@marcosluis2186 <http://twitter.com/marcosluis2186>

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB