Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Sane max storage size for DN


Copy link to this message
-
Re: Sane max storage size for DN
Thank you so much for the valuable response Ted.

No, there would be dedicated storage for NN as well.

Any tips on RAM & N/W?

*Computations are not really frequent.

Thanks again.

Regards,
    Mohammad Tariq

On Wed, Dec 12, 2012 at 9:14 PM, Ted Dunning <[EMAIL PROTECTED]> wrote:

>
> Yes it does make sense, depending on how much compute each byte of data
> will require on average.  With ordinary Hadoop, it is reasonable to have
> half a dozen 2TB drives.  With specialized versions of Hadoop considerably
> more can be supported.
>
> From what you say, it sounds like you are suggesting that your name node
> get a part of a single drive with the rest being shared with other virtual
> instances or with an OS partition.  That would be a really bad idea for
> performance.  Many Hadoop programs are I/O bound so having more than one
> spindle is a good thing.
>
>
>
> On Wed, Dec 12, 2012 at 7:02 AM, Mohammad Tariq <[EMAIL PROTECTED]>wrote:
>
>> Hello list,
>>
>>           I don't know if this question makes any sense, but I would like
>> to ask, does it make sense to store 500TB (or more) data in a single DN?If
>> yes, then what should be the spec of other parameters *viz*. NN & DN
>> RAM, N/W etc?If no, what could be the alternative?
>>
>> Many thanks.
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB