-Re: Question about disk space allocation in hadoop
Yu Li 2010-06-30, 13:33
Hi Steve and Vitaliy,
Thanks a lot for your answers, and thanks for Vitaliy's suggestion, I'll
send questions to relevant mailing list:)
2010/6/30 Vitaliy Semochkin <[EMAIL PROTECTED]>
> set dfs.datanode.du.reserved to amount of bytes you want to reserver for
> HDFS usage.
> for search convenience IMHO better post such questions to
> [EMAIL PROTECTED] ;-)
> Vitaliy S
> On Tue, Jun 29, 2010 at 8:32 AM, Yu Li <[EMAIL PROTECTED]> wrote:
> > Hi all,
> > As we all know, machines in hadoop cluster may be both datanode and
> > tasktracker, so one machine may store both MR job intermediate data
> > and HDFS data. My question is: if we have more than one disk per node,
> > say 4 disks, and would like both job intermediate data and HDFS data
> > store into all disks to reduce IO times of each single disk, can we
> > draw a line between space of local FS and HDFS? For example, restrict
> > the intermediate temp data occupy no more than 25% space on each disk?
> > Thanks in advance.
> > Best Regards,
> > Carp