Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> Can hadoop.tmp.dir be multivalued?

Copy link to this message
Can hadoop.tmp.dir be multivalued?
Hi All,

On my worker nodes i have 10 drives. So, in order to balance disk i/o i
wanted to evenly distribute the disk read/write load. "hadoop.tmp.dir" is
used for a lot of things in MR.

mapreduce.cluster.local.dir${hadoop.tmp.dir}/mapred/localThe local
directory where MapReduce stores intermediate data files. May be a
comma-separated list of directories on different devices in order to spread
disk i/o. Directories that do not exist are ignored.
mapreduce.jobtracker.system.dir${hadoop.tmp.dir}/mapred/systemThe directory
where MapReduce stores control files.  mapreduce.jobtracker.staging.root.dir
${hadoop.tmp.dir}/mapred/stagingThe root of the staging area for users' job
files In practice, this should be the directory where users' home
directories are located (usually /user)  mapreduce.cluster.temp.dir
${hadoop.tmp.dir}/mapred/tempA shared directory for temporary files.
I am aware that mapreduce.cluster.local.dir can be multivalued and i can
exlicitly set this property but i was wondering that it would be even
better if i can set multiple values in hadoop.tmp.dir property. Also,
is mapreduce.cluster.temp.dir
property multivalued or single valued?

Thanks & Regards,
Anil Gupta
Harsh J 2012-12-18, 19:13