Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Question about disk space allocation in hadoop


Copy link to this message
-
Re: Question about disk space allocation in hadoop
Yu Li 2010-06-30, 03:10
Hi all,

Anybody has experience on this? Any Comments/Suggestions would be
highly appreciated, Thanks.

Best Regards,
Carp

2010/6/29 Yu Li <[EMAIL PROTECTED]>:
> Hi all,
>
> As we all know, machines in hadoop cluster may be both datanode and
> tasktracker, so one machine may store both MR job intermediate data
> and HDFS data. My question is: if we have more than one disk per node,
> say 4 disks, and would like both job intermediate data and HDFS data
> store into all disks to reduce IO times of each single disk, can we
> draw a line between space of local FS and HDFS? For example, restrict
> the intermediate temp data occupy no more than 25% space on each disk?
> Thanks in advance.
>
> Best Regards,
> Carp
>