Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Sane max storage size for DN

Copy link to this message
Re: Sane max storage size for DN
Yes it does make sense, depending on how much compute each byte of data
will require on average.  With ordinary Hadoop, it is reasonable to have
half a dozen 2TB drives.  With specialized versions of Hadoop considerably
more can be supported.

>From what you say, it sounds like you are suggesting that your name node
get a part of a single drive with the rest being shared with other virtual
instances or with an OS partition.  That would be a really bad idea for
performance.  Many Hadoop programs are I/O bound so having more than one
spindle is a good thing.

On Wed, Dec 12, 2012 at 7:02 AM, Mohammad Tariq <[EMAIL PROTECTED]> wrote:

> Hello list,
>           I don't know if this question makes any sense, but I would like
> to ask, does it make sense to store 500TB (or more) data in a single DN?If
> yes, then what should be the spec of other parameters *viz*. NN & DN RAM,
> N/W etc?If no, what could be the alternative?
> Many thanks.
> Regards,
>     Mohammad Tariq