Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> 2.0.3-alpha in pseudodristributed mode: How to force examples to run on yarn?


Copy link to this message
-
2.0.3-alpha in pseudodristributed mode: How to force examples to run on yarn?
Dear all,

I finally have set up 2.0.3-alpha in pseudo distributed mode as per
http://hadoop.apache.org/docs/current/hadoop-project-dist/hadoop-common/SingleCluster.html

This is evidenced by having yarn resourcemanager and nodemanager running,
no exceptions in the logs and I am able to access their web interfaces. The
node manager has registered in the resourcemanager.

Instead of setting all the listed environment variables, I opted for
setting HADOOP_HOME correctly, and used the supplied start-yarn.sh script.

When running randomwriter as indicated on the above website, I notice that
the local job runner is used, which means only one map task is ever used.

Am I missing a configuration setting to have examples run on yarn?
Is there a way to fail Jobs, when they are using LocalJobRunner?

Thanks in advance,

Jens
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB