Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> DataNode Hardware


Copy link to this message
-
Re: DataNode Hardware
Inline.
On Thursday, July 12, 2012 at 12:56 PM, Bartosz M. Frak wrote:

> Quick question about data node hadrware. I've read a few articles, which
> cover the basics, including the Cloudera's recommendations here:
> http://www.cloudera.com/blog/2010/03/clouderas-support-team-shares-some-basic-hardware-recommendations/
>
> The article is from early 2010, but I'm assuming that the general
> guidelines haven't deviated much from the recommended baselines. I'm
> skewing my build towards the "Compute optimized" side of the spectrum,
> which calls for a a 1:1 core to spindle model and more RAM for per node
> for in-memory caching.
>
>

Why are you skewing more towards compute optimized. Are you expecting to run compute intensive MR interacting with HBase tables?
> Other important consideration is low(ish) power
> consumption. With that in mind I had specced out the following (per node):
>
> Chassis: 1U Supermicro chassis with 2x 1Gb/sec ethernet ports
> (http://www.supermicro.com/products/system/1u/5017/sys-5017c-mtf.cfm)
> (~500USD)
> Memory: 32GB Unbuffered ECC RAM (~280USD)
> Disks: 4x2TBHitachi Ultrastar 7200RPM SAS Drives (~960USD)
>
>

You can use plain SATA. Don't need SAS.
> CPU: 1x Intel E3-1230-v2 (3.3Ghz 4 Core / 8 Thread 69W) (~240USD)
>
>

Consider getting dual hex core CPUs.
>
> The backplane will consist of a dedicated high powered switch (not sure
> which one yet) with each node utilizing link aggregation.
>
> Does this look reasonable? We are looking into buying 4-5 of those for
> our initial test bench for under $10000 and plan to expand to about
> 50-100 nodes by next year.
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB