Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> cannot submit a job via java client in hadoop- 2.0.5-alpha


Copy link to this message
-
cannot submit a job via java client in hadoop- 2.0.5-alpha
Hi,All

 

I have a hadoop- 2.0.5-alpha cluster with 3 data nodes . I have Resource
Manager and all data nodes started and can access web ui of Resource
Manager.

I wrote a java client to submit a job as TestJob class below. But the job is
never submitted successfully. It throws out exception all the time.

My configurations are attached.  Can anyone help me? Thanks.

 

---------my-java client

public class TestJob {

    

    public void execute() {

 

        Configuration conf1 = new Configuration();

        conf1.addResource("resources/core-site.xml");

        conf1.addResource("resources/hdfs-site.xml");

        conf1.addResource("resources/yarn-site.xml");

        conf1.addResource("resources/mapred-site.xml");

        JobConf conf = new JobConf(conf1);

        

        conf.setJar("/home/francis/hadoop-jobs/MapReduceJob.jar");

        conf.setJobName("Test");

 

        conf.setInputFormat(TextInputFormat.class);

        conf.setOutputFormat(TextOutputFormat.class);

 

        conf.setOutputKeyClass(Text.class);

        conf.setOutputValueClass(IntWritable.class);

 

        conf.setMapperClass(DisplayRequestMapper.class);

        conf.setReducerClass(DisplayRequestReducer.class);

 

        FileInputFormat.setInputPaths(conf,new
Path("/home/francis/hadoop-jobs/2013070907.FNODE.2.txt"));

        FileOutputFormat.setOutputPath(conf, new
Path("/home/francis/hadoop-jobs/result/"));

 

        try {

            JobClient client = new JobClient(conf);

            RunningJob job = client.submitJob(conf);

            job.waitForCompletion();

        } catch (IOException e) {

            e.printStackTrace();

        }

    }

}

 

----------Exception

 

jvm 1    | java.io.IOException: Cannot initialize Cluster. Please check your
configuration for mapreduce.framework.name and the correspond server
addresses.

jvm 1    |      at
org.apache.hadoop.mapreduce.Cluster.initialize(Cluster.java:119)

jvm 1    |      at
org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:81)

jvm 1    |      at
org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:74)

jvm 1    |      at
org.apache.hadoop.mapred.JobClient.init(JobClient.java:482)

jvm 1    |      at
org.apache.hadoop.mapred.JobClient.<init>(JobClient.java:461)

jvm 1    |      at
com.rh.elastic.hadoop.job.TestJob.execute(TestJob.java:59)

 

 

Thanks,

Francis.Hu

 

+
Azuryy Yu 2013-07-10, 10:51
+
Devaraj k 2013-07-10, 11:10
+
Devaraj k 2013-07-10, 11:21
+
hadoop hive 2013-07-10, 10:39