Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Can't set stream.addenvironment in job configuration file


Copy link to this message
-
Can't set stream.addenvironment in job configuration file
Hi,
I'm trying to submit a streaming job using the -conf option to specify
  
a job configuration file.
One of the options in my configuration file is stream.addenvironment  
but this option doesn't appear to be getting set; i.e when I run the  
job and use the web interface to look at job.xml this property is  
missing.
Does anybody have any ideas?
My jobconf file is:
<configuration>
<property><name>stream.addenvironment</name><value>"dumbo_mrbase_class=dumbo.backends.common.MapRedBase dumbo_jk_class=dumbo.backends.common.JoinKey dumbo_runinfo_class=dumbo.backends.streaming.StreamingRunInfo  
PYTHONPATH=common.pyc:typedbytes-0.3.6-py2.6.egg"</value></property>
<property><name>stream.map.input</name><value>typedbytes</value></property>
<property><name>stream.reduce.input</name><value>typedbytes</value></property>
<property><name>stream.map.output</name><value>typedbytes</value></property>
<property><name>stream.reduce.output</name><value>typedbytes</value></property>
<property><name>mapred.job.name</name><value>group_clusters.py  
(1/1)</value></property>
</configuration>

Is it possible that the value I set for stream.addenvironment in the
job configuration file is overwritten by the call
jobConf._set("stream.addenvironment",addTaskEnvironment_);
in StreamJob.setJobConf()?

I'm using CDH3B.
Thanks
Jeremy
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB