Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - cannot submit a job via java client in hadoop- 2.0.5-alpha


Copy link to this message
-
Re: cannot submit a job via java client in hadoop- 2.0.5-alpha
Azuryy Yu 2013-07-10, 10:51
you didn't set yarn.nodemanager.address in your yarn-site.xml
On Wed, Jul 10, 2013 at 4:33 PM, Francis.Hu <[EMAIL PROTECTED]>wrote:

>  Hi,All****
>
> ** **
>
> I have a hadoop- 2.0.5-alpha cluster with 3 data nodes . I have Resource
> Manager and all data nodes started and can access web ui of Resource
> Manager. ****
>
> I wrote a java client to submit a job as TestJob class below. But the job
> is never submitted successfully. It throws out exception all the time. ***
> *
>
> My configurations are attached.  Can anyone help me? Thanks.****
>
> ** **
>
> ---------my-java client****
>
> *public* *class* TestJob {****
>
>     ****
>
>     *public* *void* execute() {****
>
> ** **
>
>         Configuration conf1 = *new* Configuration();****
>
>         conf1.addResource("resources/core-site.xml");****
>
>         conf1.addResource("resources/hdfs-site.xml");****
>
>         conf1.addResource("resources/yarn-site.xml");****
>
>         conf1.addResource("resources/mapred-site.xml");****
>
>         JobConf conf = *new* JobConf(conf1);****
>
>         ****
>
>         conf.setJar("/home/francis/hadoop-jobs/MapReduceJob.jar");****
>
>         conf.setJobName("Test");****
>
> ** **
>
>         conf.setInputFormat(TextInputFormat.*class*);****
>
>         conf.setOutputFormat(TextOutputFormat.*class*);****
>
> ** **
>
>         conf.setOutputKeyClass(Text.*class*);****
>
>         conf.setOutputValueClass(IntWritable.*class*);****
>
> ** **
>
>         conf.setMapperClass(DisplayRequestMapper.*class*);****
>
>         conf.setReducerClass(DisplayRequestReducer.*class*);****
>
> ** **
>
>         FileInputFormat.*setInputPaths*(conf,*new* Path(
> "/home/francis/hadoop-jobs/2013070907.FNODE.2.txt"));****
>
>         FileOutputFormat.*setOutputPath*(conf, *new* Path(
> "/home/francis/hadoop-jobs/result/"));****
>
> ** **
>
>         *try* {****
>
>             JobClient client = *new* JobClient(conf);****
>
>             RunningJob job = client.submitJob(conf);****
>
>             job.waitForCompletion();****
>
>         } *catch* (IOException e) {****
>
>             e.printStackTrace();****
>
>         }****
>
>     }****
>
> }****
>
> ** **
>
> ----------Exception ****
>
> ** **
>
> jvm 1    | java.io.IOException: Cannot initialize Cluster. Please check
> your configuration for mapreduce.framework.name and the correspond server
> addresses.****
>
> jvm 1    |      at
> org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:119)****
>
> jvm 1    |      at
> org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:81)****
>
> jvm 1    |      at
> org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74)****
>
> jvm 1    |      at
> org.apache.hadoop.mapred.JobClient.init(JobClient.java:482)****
>
> jvm 1    |      at
> org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:461)****
>
> jvm 1    |      at
> com.rh.elastic.hadoop.job.TestJob.execute(TestJob.java:59)****
>
> ** **
>
> ** **
>
> Thanks,****
>
> Francis.Hu****
>
> ** **
>