Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re: Which hardware to choose


+
Oleg Ruchovets 2012-10-02, 14:59
+
Alexander Pivovarov 2012-10-02, 18:55
+
hadoopman 2012-10-03, 01:14
+
Michael Segel 2012-10-03, 01:40
Copy link to this message
-
Re: Which hardware to choose
Which is a reasonable number in this hardware?

On 10/02/2012 09:40 PM, Michael Segel wrote:
> I think he's saying that its 24 maps 8 reducers per node and at 48GB that could be too many mappers.
> Especially if they want to run HBase.
>
> On Oct 2, 2012, at 8:14 PM, hadoopman <[EMAIL PROTECTED]> wrote:
>
>> Only 24 map and 8 reduce tasks for 38 data nodes?  are you sure that's right?  Sounds VERY low for a cluster that size.
>>
>> We have only 10 c2100's and are running I believe 140 map and 70 reduce slots so far with pretty decent performance.
>>
>>
>>
>> On 10/02/2012 12:55 PM, Alexander Pivovarov wrote:
>>> 38 data nodes + 2 Name Nodes
>>>>   >
>>>>   >  Data Node:
>>>>   >  Dell PowerEdge C2100 series
>>>>   >  2 x XEON x5670
>>>>   >  48 GB RAM ECC  (12x4GB 1333MHz)
>>>>   >  12 x 2 TB  7200 RPM SATA HDD (with hot swap)  JBOD
>>>>   >  Intel Gigabit ET Dual port PCIe x4
>>>>   >  Redundant Power Supply
>>>>   >  Hadoop CDH3
>>>>   >  max map tasks 24
>>>>   >  max reduce tasks 8
>>
>
> 10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
> CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION
>
> http://www.uci.cu
> http://www.facebook.com/universidad.uci
> http://www.flickr.com/photos/universidad_uci

--

Marcos Luis Ort�z Valmaseda
*Data Engineer && Sr. System Administrator at UCI*
about.me/marcosortiz <http://about.me/marcosortiz>
My Blog <http://marcosluis2186.posterous.com>
Tumblr's blog <http://marcosortiz.tumblr.com/>
@marcosluis2186 <http://twitter.com/marcosluis2186>

10mo. ANIVERSARIO DE LA CREACION DE LA UNIVERSIDAD DE LAS CIENCIAS INFORMATICAS...
CONECTADOS AL FUTURO, CONECTADOS A LA REVOLUCION

http://www.uci.cu
http://www.facebook.com/universidad.uci
http://www.flickr.com/photos/universidad_uci
+
Michael Segel 2012-10-03, 02:35
+
Alexander Pivovarov 2012-10-03, 03:42
+
J. Rottinghuis 2012-10-03, 17:09
+
Michael Segel 2012-10-03, 17:21
+
Russell Jurney 2012-10-03, 01:19
+
hadoopman 2012-10-03, 01:33