Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # user >> Re:


For a job to get submitted to a cluster, you will need proper client
configurations. Have you configured your mapred-site.xml and
yarn-site.xml properly inside /etc/hadoop/conf/mapred-site.xml and
/etc/hadoop/conf/yarn-site.xml at the client node?

On Mon, Jul 30, 2012 at 12:00 AM, abhiTowson cal
<[EMAIL PROTECTED]> wrote:
> Hi All,
>
> I am getting problem that job is running in localrunner rather than
> the cluster enviormnent.
> And when am running the job i would not be able to see the job id in
> the resource manager UI
>
> Can you please go through the issues and let me know ASAP.
>
> sudo -u hdfs hadoop jar
> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples.jar teragen
> 1000000 /benchmark/teragen/input
> 12/07/29 13:35:59 WARN conf.Configuration: session.id is deprecated.
> Instead, use dfs.metrics.session-id
> 12/07/29 13:35:59 INFO jvm.JvmMetrics: Initializing JVM Metrics with
> processName=JobTracker, sessionId> 12/07/29 13:35:59 INFO util.NativeCodeLoader: Loaded the native-hadoop library
> 12/07/29 13:35:59 WARN mapred.JobClient: Use GenericOptionsParser for
> parsing the arguments. Applications should implement Tool for the
> same.
> Generating 1000000 using 1 maps with step of 1000000
> 12/07/29 13:35:59 INFO mapred.JobClient: Running job: job_local_0001
> 12/07/29 13:35:59 INFO mapred.LocalJobRunner: OutputCommitter set in config null
> 12/07/29 13:35:59 INFO mapred.LocalJobRunner: OutputCommitter is
> org.apache.hadoop.mapred.FileOutputCommitter
> 12/07/29 13:35:59 WARN mapreduce.Counters: Group
> org.apache.hadoop.mapred.Task$Counter is deprecated. Use
> org.apache.hadoop.mapreduce.TaskCounter instead
> 12/07/29 13:35:59 INFO util.ProcessTree: setsid exited with exit code 0
> 12/07/29 13:35:59 INFO mapred.Task:  Using ResourceCalculatorPlugin :
> org.apache.hadoop.util.LinuxResourceCalculatorPlugin@47c297a3
> 12/07/29 13:36:00 WARN mapreduce.Counters: Counter name
> MAP_INPUT_BYTES is deprecated. Use FileInputFormatCounters as group
> name and  BYTES_READ as counter name instead
> 12/07/29 13:36:00 INFO mapred.MapTask: numReduceTasks: 0
> 12/07/29 13:36:00 INFO mapred.JobClient:  map 0% reduce 0%
> 12/07/29 13:36:01 INFO mapred.Task: Task:attempt_local_0001_m_000000_0
> is done. And is in the process of commiting
> 12/07/29 13:36:01 INFO mapred.LocalJobRunner:
> 12/07/29 13:36:01 INFO mapred.Task: Task attempt_local_0001_m_000000_0
> is allowed to commit now
> 12/07/29 13:36:01 INFO mapred.FileOutputCommitter: Saved output of
> task 'attempt_local_0001_m_000000_0' to
> hdfs://hadoop-master-1/benchmark/teragen/input
> 12/07/29 13:36:01 INFO mapred.LocalJobRunner:
> 12/07/29 13:36:01 INFO mapred.Task: Task 'attempt_local_0001_m_000000_0' done.
> 12/07/29 13:36:02 INFO mapred.JobClient:  map 100% reduce 0%
> 12/07/29 13:36:02 INFO mapred.JobClient: Job complete: job_local_0001
> 12/07/29 13:36:02 INFO mapred.JobClient: Counters: 19
> 12/07/29 13:36:02 INFO mapred.JobClient:   File System Counters
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of bytes read=142686
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of bytes
> written=220956
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of read operations=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of large
> read operations=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     FILE: Number of write operations=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of bytes read=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of bytes
> written=100000000
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of read operations=1
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of large
> read operations=0
> 12/07/29 13:36:02 INFO mapred.JobClient:     HDFS: Number of write operations=2
> 12/07/29 13:36:02 INFO mapred.JobClient:   Map-Reduce Framework
> 12/07/29 13:36:02 INFO mapred.JobClient:     Map input records=1000000
> 12/07/29 13:36:02 INFO mapred.JobClient:     Map output records=1000000

Harsh J
+
anil gupta 2012-07-29, 19:05
+
abhiTowson cal 2012-07-29, 19:12
+
Anil Gupta 2012-07-29, 19:17
+
abhiTowson cal 2012-07-29, 19:20
+
anil gupta 2012-07-29, 19:28
+
abhiTowson cal 2012-07-29, 19:23
+
anil gupta 2012-07-29, 19:31
+
abhiTowson cal 2012-07-29, 20:08
+
anil gupta 2012-07-29, 22:47
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB