Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # user - Re:


Copy link to this message
-
Re:
Harsh J 2012-07-29, 18:43
For a job to get submitted to a cluster, you will need proper client
configurations. Have you configured your mapred-site.xml and
yarn-site.xml properly inside /etc/hadoop/conf/mapred-site.xml and
/etc/hadoop/conf/yarn-site.xml at the client node?

On Mon, Jul 30, 2012 at 12:00 AM, abhiTowson cal
<[EMAIL PROTECTED]> wrote:
> Hi All,
>
> I am getting problem that job is running in localrunner rather than
> the cluster enviormnent.
> And when am running the job i would not be able to see the job id in
> the resource manager UI
>
> Can you please go through the issues and let me know ASAP.
>
> sudo -u hdfs hadoop jar
> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen
> 1000000 /benchmark/teragen/input
> 12/07/29 13:35:59 WARN conf.Configuration: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> 12/07/29 13:35:59 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId> 12/07/29 13:35:59 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/07/29 13:35:59 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the
> same.
> Generating 1000000 using 1 maps with step of 1000000
> 12/07/29 13:35:59 INFO mapred.JobClient: Running job: job_local_0001
> 12/07/29 13:35:59 INFO mapred.LocalJobRunner: OutputCommitter set in config null
> 12/07/29 13:35:59 INFO mapred.LocalJobRunner: OutputCommitter is
> org.apache.hadoop.mapred.FileOutputCommitter
> 12/07/29 13:35:59 WARN mapreduce.Counters: Group
> org.apache.hadoop.mapred.Task$Counter is deprecated. Use
> org.apache.hadoop.mapreduce.TaskCounter instead
> 12/07/29 13:35:59 INFO util.ProcessTree: setsid exited with exit code 0
> 12/07/29 13:35:59 INFO mapred.Task:  Using ResourceCalculatorPlugin :
> org.apache.hadoop.util.LinuxResourceCalculatorPlugin@47c297a3
> 12/07/29 13:36:00 WARN mapreduce.Counters: Counter name
> MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group
> name and  BYTES_READ as counter name instead
> 12/07/29 13:36:00 INFO mapred.MapTask: numReduceTasks: 0
> 12/07/29 13:36:00 INFO mapred.JobClient:  map 0% reduce 0%
> 12/07/29 13:36:01 INFO mapred.Task: Task:attempt_local_0001_m_000000_0
> is done. And is in the process of commiting
> 12/07/29 13:36:01 INFO mapred.LocalJobRunner:
> 12/07/29 13:36:01 INFO mapred.Task: Task attempt_local_0001_m_000000_0
> is allowed to commit now
> 12/07/29 13:36:01 INFO mapred.FileOutputCommitter: Saved output of
> task 'attempt_local_0001_m_000000_0' to
> hdfs://hadoop-master-1/benchmark/teragen/input
> 12/07/29 13:36:01 INFO mapred.LocalJobRunner:
> 12/07/29 13:36:01 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0' done.
> 12/07/29 13:36:02 INFO mapred.JobClient:  map 100% reduce 0%
> 12/07/29 13:36:02 INFO mapred.JobClient: Job complete: job_local_0001
> 12/07/29 13:36:02 INFO mapred.JobClient: Counters: 19
> 12/07/29 13:36:02 INFO mapred.JobClient:   File System Counters
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of bytes read=142686
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of bytes
> written=220956
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of read operations=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of large
> read operations=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of write operations=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of bytes read=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of bytes
> written=100000000
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of read operations=1
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of large
> read operations=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of write operations=2
> 12/07/29 13:36:02 INFO mapred.JobClient:   Map-Reduce Framework
> 12/07/29 13:36:02 INFO mapred.JobClient:     Map input records=1000000
> 12/07/29 13:36:02 INFO mapred.JobClient:     Map output records=1000000

Harsh J
+
anil gupta 2012-07-29, 19:05
+
abhiTowson cal 2012-07-29, 19:12
+
Anil Gupta 2012-07-29, 19:17
+
abhiTowson cal 2012-07-29, 19:20
+
anil gupta 2012-07-29, 19:28
+
abhiTowson cal 2012-07-29, 19:23
+
anil gupta 2012-07-29, 19:31
+
abhiTowson cal 2012-07-29, 20:08
+
anil gupta 2012-07-29, 22:47