Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: Hardware Selection for Hadoop


Copy link to this message
-
Re: Hardware Selection for Hadoop
If I were to start with a 5 node cluster, I would do this :

*Machine 1 : *NN+JT
32GB RAM, 2xQuad Core Proc, 500GB SATA HDD along with a NAS(To make sure
metadata is safe)

*Machine 2 : *SNN*
*
32GB RAM, 2xQuad Core Proc, 500GB SATA HDD

*Machine 3,4,5 : *DN+TT
16GB RAM, 2xQuad Core Proc, 5 x 200GB SATA HDD(JBOD configuation)

I don't think you'll require 64GB RAM and so much of storage just for a
POC(but, it actually depends). You can really kick the ass with 32GB.

NIC(network interface card) is a computer hardware component that connects
a computer to a computer network. It must be reliable to make sure that all
your machines are always connected to the cluster and there is no problem
in the data transfer.

Apart from this ask them to provide you cabinets with good ventilation and
cooling mechanism.

Warm Regards,
Tariq
https://mtariq.jux.com/
cloudfront.blogspot.com
On Tue, Apr 30, 2013 at 2:17 AM, Raj Hadoop <[EMAIL PROTECTED]> wrote:

> Hi,
>
> In 5 node cluster - you mean
>
> Name Node , Job Tracker , Secondary Name Node all on 1
>         64 GB Ram ( Processor - 2 x Quad cores Intel  , Storage - ? )
>
> Data Trackers and Job Trackers - on 4 machies - each of
>         32 GB Ram ( Processor - 2 x Quad cores Intel  , Storage - ? )
>
> NIC ?
>
> Also - what other details should I provide to my hardware engineer.
>
> The idea is to start with a Web Log Processing proof of concept.
>
> Please advise.
>
>
>   *From:* Patai Sangbutsarakum <[EMAIL PROTECTED]>
> *To:* "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> *Sent:* Monday, April 29, 2013 2:49 PM
> *Subject:* Re: Hardware Selection for Hadoop
>
>  2 x Quad cores Intel
> 2-3 TB x 6 SATA
> 64GB mem
> 2 NICs teaming
>
> my 2 cents
>
>
>  On Apr 29, 2013, at 9:24 AM, Raj Hadoop <[EMAIL PROTECTED]>
>  wrote:
>
>      Hi,
>
> I have to propose some hardware requirements in my company for a Proof of
> Concept with Hadoop. I was reading Hadoop Operations and also saw Cloudera
> Website. But just wanted to know from the group - what is the requirements
> if I have to plan for a 5 node cluster. I dont know at this time, the data
> that need to be processed at this time for the Proof of Concept. So - can
> you suggest something to me?
>
> Regards,
> Raj
>
>
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB