Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Question about disk space allocation in hadoop


Copy link to this message
-
Re: Question about disk space allocation in hadoop
Hi Steve and Vitaliy,

Thanks a lot for your answers, and thanks for Vitaliy's suggestion, I'll
send questions to relevant mailing list:)

Best Regards,
Carp
2010/6/30 Vitaliy Semochkin <[EMAIL PROTECTED]>

> set dfs.datanode.du.reserved to amount of bytes you want to reserver for
> not
> HDFS usage.
>
> PS
> for search convenience IMHO better post such questions to
> [EMAIL PROTECTED] ;-)
>
>
> Regards,
> Vitaliy S
>
> On Tue, Jun 29, 2010 at 8:32 AM, Yu Li <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
> >
> > As we all know, machines in hadoop cluster may be both datanode and
> > tasktracker, so one machine may store both MR job intermediate data
> > and HDFS data. My question is: if we have more than one disk per node,
> > say 4 disks, and would like both job intermediate data and HDFS data
> > store into all disks to reduce IO times of each single disk, can we
> > draw a line between space of local FS and HDFS? For example, restrict
> > the intermediate temp data occupy no more than 25% space on each disk?
> > Thanks in advance.
> >
> > Best Regards,
> > Carp
> >
>