Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Can hadoop.tmp.dir be multivalued?


Copy link to this message
-
Can hadoop.tmp.dir be multivalued?
Hi All,

On my worker nodes i have 10 drives. So, in order to balance disk i/o i
wanted to evenly distribute the disk read/write load. "hadoop.tmp.dir" is
used for a lot of things in MR.

mapreduce.cluster.local.dir${hadoop.tmp.dir}/mapred/localThe local
directory where MapReduce stores intermediate data files. May be a
comma-separated list of directories on different devices in order to spread
disk i/o. Directories that do not exist are ignored.
mapreduce.jobtracker.system.dir${hadoop.tmp.dir}/mapred/systemThe directory
where MapReduce stores control files.  mapreduce.jobtracker.staging.root.dir
${hadoop.tmp.dir}/mapred/stagingThe root of the staging area for users' job
files In practice, this should be the directory where users' home
directories are located (usually /user)  mapreduce.cluster.temp.dir
${hadoop.tmp.dir}/mapred/tempA shared directory for temporary files.
I am aware that mapreduce.cluster.local.dir can be multivalued and i can
exlicitly set this property but i was wondering that it would be even
better if i can set multiple values in hadoop.tmp.dir property. Also,
is mapreduce.cluster.temp.dir
property multivalued or single valued?

--
Thanks & Regards,
Anil Gupta
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB