Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Can't set stream.addenvironment in job configuration file


Copy link to this message
-
Can't set stream.addenvironment in job configuration file
Jeremy Lewi 2011-04-08, 04:35
Hi,
I'm trying to submit a streaming job using the -conf option to specify
  
a job configuration file.
One of the options in my configuration file is stream.addenvironment  
but this option doesn't appear to be getting set; i.e when I run the  
job and use the web interface to look at job.xml this property is  
missing.
Does anybody have any ideas?
My jobconf file is:
<configuration>
<property><name>stream.addenvironment</name><value>"dumbo_mrbase_class=dumbo.backends.common.MapRedBase dumbo_jk_class=dumbo.backends.common.JoinKey dumbo_runinfo_class=dumbo.backends.streaming.StreamingRunInfo  
PYTHONPATH=common.pyc:typedbytes-0.3.6-py2.6.egg"</value></property>
<property><name>stream.map.input</name><value>typedbytes</value></property>
<property><name>stream.reduce.input</name><value>typedbytes</value></property>
<property><name>stream.map.output</name><value>typedbytes</value></property>
<property><name>stream.reduce.output</name><value>typedbytes</value></property>
<property><name>mapred.job.name</name><value>group_clusters.py  
(1/1)</value></property>
</configuration>

Is it possible that the value I set for stream.addenvironment in the
job configuration file is overwritten by the call
jobConf._set("stream.addenvironment",addTaskEnvironment_);
in StreamJob.setJobConf()?

I'm using CDH3B.
Thanks
Jeremy