Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Question about disk space allocation in hadoop


Copy link to this message
-
Re: Question about disk space allocation in hadoop
Hi Steve and Vitaliy,

Thanks a lot for your answers, and thanks for Vitaliy's suggestion, I'll
send questions to relevant mailing list:)

Best Regards,
Carp
2010/6/30 Vitaliy Semochkin <[EMAIL PROTECTED]>

> set dfs.datanode.du.reserved to amount of bytes you want to reserver for
> not
> HDFS usage.
>
> PS
> for search convenience IMHO better post such questions to
> [EMAIL PROTECTED] ;-)
>
>
> Regards,
> Vitaliy S
>
> On Tue, Jun 29, 2010 at 8:32 AM, Yu Li <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
> >
> > As we all know, machines in hadoop cluster may be both datanode and
> > tasktracker, so one machine may store both MR job intermediate data
> > and HDFS data. My question is: if we have more than one disk per node,
> > say 4 disks, and would like both job intermediate data and HDFS data
> > store into all disks to reduce IO times of each single disk, can we
> > draw a line between space of local FS and HDFS? For example, restrict
> > the intermediate temp data occupy no more than 25% space on each disk?
> > Thanks in advance.
> >
> > Best Regards,
> > Carp
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB