Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # dev >> Problem while submitting jobs to NM started with ephemeral ports.


Copy link to this message
-
Problem while submitting jobs to NM started with ephemeral ports.
I am using following properties in yarn-site

<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce.shuffle</value>
</property>
 <property>
<name>yarn.nodemanager.aux-services.mapreduce.shuffle.class</name>
<value>org.apache.hadoop.mapred.ShuffleHandler</value>
</property>
  <property>
    <name>yarn.nodemanager.address</name>
    <value>localhost:0</value>
  </property>
  <property>
    <name>yarn.nodemanager.localizer.address</name>
    <value>localhost:0</value>
  </property>

Everything runs fine. (means all daemons are started perfectly) But
when you try to submit the job. Job is stuck and NM logs says trying
to connect to 'localhost:0'. Localization takes forever. Why?

Please see the NM logs below.

http://pastebin.com/QfQDZeqF

Thanks,
Prashant
+
Prashant Sharma 2011-10-17, 09:05
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB