Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> M/R Staticstics


Copy link to this message
-
Re: M/R Staticstics
Have you enabled security?

can you share the output for your hdfs?

bin/hadoop fs -ls /

and is /tmp/hadoop-yarn/staging/history/done directory present in hdfs ? if
so then what permissions?

also please share exception stack trace...

Thanks,
Omkar Joshi
Hortonworks Inc
On Fri, Apr 26, 2013 at 3:05 PM, <[EMAIL PROTECTED]> wrote:

>
> I was able to overcome the permission exception in the log by creating an
> HDFS tmp folder (hadoop fs -mkdir /tmp) and opening it up to the world
> (hadoop fs -chmod a+rwx /tmp). That got rid of the exception put I still am
> able to connect to port 50030 to see M/R status. More ideas?
>
> Even though the exception was missing from the logs of one server in the
> cluster, l looked on another server and found essentially the same
> permission problem:
>
> 2013-04-26 13:34:56,462 FATAL
> org.apache.hadoop.mapreduce.v2.hs.JobHistoryServer: Error starting
> JobHistoryServer
> org.apache.hadoop.yarn.YarnException: Error creating done directory:
> [hdfs://devubuntu05:9000/tmp/hadoop-yarn/staging/history/done]
>         at
> org.apache.hadoop.mapreduce.v2.hs.HistoryFileManager.init(HistoryFileManager.java:424)
>         at
> org.apache.hadoop.mapreduce.v2.hs.JobHistory.init(JobHistory.java:87)
>         at
> org.apache.hadoop.yarn.service.CompositeService.init(CompositeService.java:58)
>
> . . . . .
>
> On Fri, Apr 26, 2013 at 10:37 AM, Rishi Yadav wrote:
>
>   do you see "retired jobs" on job tracker page. There is also "job
> tracker history" on the bottom of page.
>
> something like this  *http://nn.zettabyte.com:50030/jobtracker.jsp*<http://nn.zettabyte.com:50030/jobtracker.jsp>
> Thanks and Regards,
> Rishi Yadav
>
>
>
>
>
> On Fri, Apr 26, 2013 at 7:36 AM, < *[EMAIL PROTECTED]*> wrote:
> When I submit a simple "Hello World" M/R job like WordCount it takes less
> than 5 seconds. The texts show numerous methods for monitoring M/R jobs as
> they are happening but I have yet to see any that show statistics about a
> job after it has completed. Obviously simple jobs that take a short amount
> of time don't allow time to fire up any web mage or monitoring tool to see
> how it progresses through the JobTracker and TaskTracker as well as which
> node it is processed on. Any suggestions on how could see this kind of data
> *after* a job has completed?
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB