Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> cannot submit a job via java client in hadoop- 2.0.5-alpha


+
Francis.Hu 2013-07-10, 08:33
Copy link to this message
-
Re: cannot submit a job via java client in hadoop- 2.0.5-alpha
you didn't set yarn.nodemanager.address in your yarn-site.xml
On Wed, Jul 10, 2013 at 4:33 PM, Francis.Hu <[EMAIL PROTECTED]>wrote:

>  Hi,All****
>
> ** **
>
> I have a hadoop- 2.0.5-alpha cluster with 3 data nodes . I have Resource
> Manager and all data nodes started and can access web ui of Resource
> Manager. ****
>
> I wrote a java client to submit a job as TestJob class below. But the job
> is never submitted successfully. It throws out exception all the time. ***
> *
>
> My configurations are attached.  Can anyone help me? Thanks.****
>
> ** **
>
> ---------my-java client****
>
> *public* *class* TestJob {****
>
>     ****
>
>     *public* *void* execute() {****
>
> ** **
>
>         Configuration conf1 = *new* Configuration();****
>
>         conf1.addResource("resources/core-site.xml");****
>
>         conf1.addResource("resources/hdfs-site.xml");****
>
>         conf1.addResource("resources/yarn-site.xml");****
>
>         conf1.addResource("resources/mapred-site.xml");****
>
>         JobConf conf = *new* JobConf(conf1);****
>
>         ****
>
>         conf.setJar("/home/francis/hadoop-jobs/MapReduceJob.jar");****
>
>         conf.setJobName("Test");****
>
> ** **
>
>         conf.setInputFormat(TextInputFormat.*class*);****
>
>         conf.setOutputFormat(TextOutputFormat.*class*);****
>
> ** **
>
>         conf.setOutputKeyClass(Text.*class*);****
>
>         conf.setOutputValueClass(IntWritable.*class*);****
>
> ** **
>
>         conf.setMapperClass(DisplayRequestMapper.*class*);****
>
>         conf.setReducerClass(DisplayRequestReducer.*class*);****
>
> ** **
>
>         FileInputFormat.*setInputPaths*(conf,*new* Path(
> "/home/francis/hadoop-jobs/2013070907.FNODE.2.txt"));****
>
>         FileOutputFormat.*setOutputPath*(conf, *new* Path(
> "/home/francis/hadoop-jobs/result/"));****
>
> ** **
>
>         *try* {****
>
>             JobClient client = *new* JobClient(conf);****
>
>             RunningJob job = client.submitJob(conf);****
>
>             job.waitForCompletion();****
>
>         } *catch* (IOException e) {****
>
>             e.printStackTrace();****
>
>         }****
>
>     }****
>
> }****
>
> ** **
>
> ----------Exception ****
>
> ** **
>
> jvm 1    | java.io.IOException: Cannot initialize Cluster. Please check
> your configuration for mapreduce.framework.name and the correspond server
> addresses.****
>
> jvm 1    |      at
> org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:119)****
>
> jvm 1    |      at
> org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:81)****
>
> jvm 1    |      at
> org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74)****
>
> jvm 1    |      at
> org.apache.hadoop.mapred.JobClient.init(JobClient.java:482)****
>
> jvm 1    |      at
> org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:461)****
>
> jvm 1    |      at
> com.rh.elastic.hadoop.job.TestJob.execute(TestJob.java:59)****
>
> ** **
>
> ** **
>
> Thanks,****
>
> Francis.Hu****
>
> ** **
>
+
Devaraj k 2013-07-10, 11:10
+
Devaraj k 2013-07-10, 11:21
+
hadoop hive 2013-07-10, 10:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB