Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> RE: M/R Staticstics


Copy link to this message
-
RE: M/R Staticstics
Answers below.

 

From: Omkar Joshi [mailto:[EMAIL PROTECTED]]
Sent: Friday, April 26, 2013 7:15 PM
To: [EMAIL PROTECTED]
Subject: Re: M/R Staticstics

 

Have you enabled security?

No

 

can you share the output for your hdfs?

 

bin/hadoop fs -ls /

 

kevin@devUbuntu05:~$ hadoop fs -ls /

Found 2 items

drwxrwxrwx   - hdfs supergroup          0 2013-04-26 13:33 /tmp

drwxr-xr-x   - hdfs supergroup          0 2013-04-19 16:40 /user

 

and is /tmp/hadoop-yarn/staging/history/done directory present in hdfs ? if
so then what permissions?

 

kevin@devUbuntu05:~$ hadoop fs -ls -R /tmp

drwxrwx---   - mapred supergroup          0 2013-04-26 13:33
/tmp/hadoop-yarn

ls: Permission denied: user=kevin, access=READ_EXECUTE,
inode="/tmp/hadoop-yarn":mapred:supergroup:drwxrwx---

 

 

kevin@devUbuntu05:~$ sudo -u hdfs hadoop fs -ls -R /tmp

[sudo] password for kevin:

drwxrwx---   - mapred supergroup          0 2013-04-26 13:33
/tmp/hadoop-yarn

drwxrwx---   - mapred supergroup          0 2013-04-26 13:33
/tmp/hadoop-yarn/staging

drwxrwx---   - mapred supergroup          0 2013-04-26 13:33
/tmp/hadoop-yarn/staging/history

drwxrwx---   - mapred supergroup          0 2013-04-26 13:33
/tmp/hadoop-yarn/staging/history/done

drwxrwxrwt   - mapred supergroup          0 2013-04-26 13:33
/tmp/hadoop-yarn/staging/history/done_intermediate

kevin@devUbuntu05:~$

 

also please share exception stack trace...

 

There is no exception now that I created /tmp on HDFS. But I still cannot
see the logs via port 50030 on the master. In other words nothing seems to
be listening on http:devubuntu05:50030. The log for map reduce looks like:

 

2013-04-26 13:35:26,107 INFO
org.apache.hadoop.mapreduce.v2.hs.HistoryClientService: Instantiated
MRClientService at devUbuntu05/172.16.26.68:10020

2013-04-26 13:35:26,107 INFO org.apache.hadoop.yarn.service.AbstractService:
Service:HistoryClientService is started.

2013-04-26 13:35:26,107 INFO org.apache.hadoop.yarn.service.AbstractService:
Service:org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer is started.

2013-04-26 13:35:55,290 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
History Cleaner started

2013-04-26 13:35:55,295 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
History Cleaner complete

2013-04-26 13:38:25,283 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
Starting scan to move intermediate done files

2013-04-26 13:41:25,283 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
Starting scan to move intermediate done files

2013-04-26 13:44:25,283 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
Starting scan to move intermediate done files

2013-04-26 13:47:25,283 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
Starting scan to move intermediate done files

2013-04-26 13:50:25,283 INFO org.apache.hadoop.mapreduce.v2.hs.JobHistory:
Starting scan to move intermediate done files

 

 

Thanks,

Omkar Joshi

Hortonworks Inc

 

On Fri, Apr 26, 2013 at 3:05 PM, <[EMAIL PROTECTED]> wrote:

  

I was able to overcome the permission exception in the log by creating an
HDFS tmp folder (hadoop fs -mkdir /tmp) and opening it up to the world
(hadoop fs -chmod a+rwx /tmp). That got rid of the exception put I still am
able to connect to port 50030 to see M/R status. More ideas?

 

Even though the exception was missing from the logs of one server in the
cluster, l looked on another server and found essentially the same
permission problem:

 

2013-04-26 13:34:56,462 FATAL
org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: Error starting
JobHistoryServer

org.apache.hadoop.yarn.YarnException: Error creating done directory:
[hdfs://devubuntu05:9000/tmp/hadoop-yarn/staging/history/done]

        at
org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.init(HistoryFileManager
.java:424)

        at
org.apache.hadoop.mapreduce.v2.hs.JobHistory.init(JobHistory.java:87)

        at
org.apache.hadoop.yarn.service.CompositeService.init(CompositeService.java:5
8)

 

. . . . .

 

On Fri, Apr 26, 2013 at 10:37 AM, Rishi Yadav wrote:

 

  do you see "retired jobs" on job tracker page. There is also "job tracker
history" on the bottom of page.  

 

something like this  http://nn.zettabyte.com:50030/jobtracker.jsp

Thanks and Regards,

Rishi Yadav

On Fri, Apr 26, 2013 at 7:36 AM, < [EMAIL PROTECTED]> wrote:

When I submit a simple "Hello World" M/R job like WordCount it takes less
than 5 seconds. The texts show numerous methods for monitoring M/R jobs as
they are happening but I have yet to see any that show statistics about a
job after it has completed. Obviously simple jobs that take a short amount
of time don't allow time to fire up any web mage or monitoring tool to see
how it progresses through the JobTracker and TaskTracker as well as which
node it is processed on. Any suggestions on how could see this kind of data
*after* a job has completed?

 

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB