Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - Can hadoop.tmp.dir be multivalued?


Copy link to this message
-
Can hadoop.tmp.dir be multivalued?
anil gupta 2012-12-18, 18:45
Hi All,

On my worker nodes i have 10 drives. So, in order to balance disk i/o i
wanted to evenly distribute the disk read/write load. "hadoop.tmp.dir" is
used for a lot of things in MR.

mapreduce.cluster.local.dir${hadoop.tmp.dir}/mapred/localThe local
directory where MapReduce stores intermediate data files. May be a
comma-separated list of directories on different devices in order to spread
disk i/o. Directories that do not exist are ignored.
mapreduce.jobtracker.system.dir${hadoop.tmp.dir}/mapred/systemThe directory
where MapReduce stores control files.  mapreduce.jobtracker.staging.root.dir
${hadoop.tmp.dir}/mapred/stagingThe root of the staging area for users' job
files In practice, this should be the directory where users' home
directories are located (usually /user)  mapreduce.cluster.temp.dir
${hadoop.tmp.dir}/mapred/tempA shared directory for temporary files.
I am aware that mapreduce.cluster.local.dir can be multivalued and i can
exlicitly set this property but i was wondering that it would be even
better if i can set multiple values in hadoop.tmp.dir property. Also,
is mapreduce.cluster.temp.dir
property multivalued or single valued?

--
Thanks & Regards,
Anil Gupta