Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: issue about hadoop hardware choose


Copy link to this message
-
Re: issue about hadoop hardware choose
Hello Ch Huang,
Do you know this book?
"Hadoop Operations" http://shop.oreilly.com/product/0636920025085.do

I think, it answers most of the questions in detail.

For a production cluster you should consider MRv1.
And I suggest you, to go with more hard drives per slave node to have a
higher
IO bandwith for map reduce, give it 4 x 2 TB at least or even 6.
At least three zookeeper servers are used.

Best wishes
Mirko

2013/8/8 ch huang <[EMAIL PROTECTED]>

> hi,all:
>             My company need build a 10 node hadoop cluster (2 namenode and
> 8 datanode & node manager ,for both data storage and data analysis ) ,we
> have hbase ,hive on the hadoop cluster, 10G data increment per day.
>             we use CDH4.3 ( for dual - namenode HA),my plan is
>
>            name node  & resource manager
>            dual Quad Core
>          24G RAM
>          2 * 500GB SATA DISK (JBOD)
>
>          datanode & node manager
>          dual Quad Core
>          24G RAM
>          2 * 1TGB SATA DISK (JBOD)
>
>
> my question is
> 1, if resource manager need a dedicated server? ( i plan to put RM with
> one of NN)
> 2, if the RAM is enough for RM + NN machine?
> 3,RAID is need for NN machine?
> 4,is it ok if i place JN on other node(DN or NN)
> 5, how much zookeeper server node i need?
> 6,i want to place yarn proxy server and mapreduce history server with
> another NN,is it ok?
>
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB