|
|
-
Re: Too many fetch-failuresAjay Srivastava 2012-12-04, 01:42
Thanks Harsh.
Problem is resolved. Entry of one of the datanodes was missing from /etc/hosts. After adding the entry, job finished without any problem. Regards, Ajay Srivastava On 04-Dec-2012, at 2:50 AM, Harsh J wrote: > What version/distribution of Hadoop is this? > > On Mon, Dec 3, 2012 at 10:51 PM, Ajay Srivastava > <[EMAIL PROTECTED]> wrote: >> Hi, >> >> I am getting following errors - >> >> 12/12/03 17:11:42 INFO mapred.JobClient: Task Id : attempt_201212031650_0001_m_000011_0, Status : FAILED >> Too many fetch-failures >> attempt_201212031650_0001_m_000011_0: WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files. >> 12/12/03 17:11:47 INFO mapred.JobClient: map 99% reduce 24% >> 12/12/03 17:12:04 INFO mapred.JobClient: map 99% reduce 32% >> 12/12/03 17:12:07 INFO mapred.JobClient: map 99% reduce 33% >> 12/12/03 17:12:13 INFO mapred.JobClient: map 99% reduce 34% >> 12/12/03 17:12:22 INFO mapred.JobClient: map 99% reduce 35% >> 12/12/03 17:12:28 INFO mapred.JobClient: map 99% reduce 36% >> 12/12/03 17:12:29 INFO mapred.JobClient: map 100% reduce 36% >> 12/12/03 17:12:39 INFO mapred.JobClient: map 100% reduce 37% >> 12/12/03 17:12:46 INFO mapred.JobClient: map 100% reduce 38% >> 12/12/03 17:12:55 INFO mapred.JobClient: map 100% reduce 39% >> 12/12/03 17:13:08 INFO mapred.JobClient: map 100% reduce 40% >> 12/12/03 17:13:23 INFO mapred.JobClient: map 100% reduce 41% >> 12/12/03 17:13:59 INFO mapred.JobClient: Task Id : attempt_201212031650_0001_m_000012_0, Status : FAILED >> Too many fetch-failures >> attempt_201212031650_0001_m_000012_0: WARNING: org.apache.hadoop.metrics.jvm.EventCounter is deprecated. Please use org.apache.hadoop.log.metrics.EventCounter in all the log4j.properties files. >> 12/12/03 17:14:04 INFO mapred.JobClient: map 99% reduce 41% >> >> I see following error in task tracker log :- >> >> 2012-12-03 17:09:40,969 INFO org.apache.hadoop.mapred.TaskTracker.clienttrace: src: 192.168.156.22:50060, dest: 192.168.156.22:42666, bytes: 131072, op: MAPRED_SHUFFLE, cliID: attempt_201212031650_0001_m_000166_0, duration: 3343330 >> 2012-12-03 17:09:40,970 ERROR org.mortbay.log: /mapOutput >> java.lang.IllegalStateException: Committed >> at org.mortbay.jetty.Response.resetBuffer(Response.java:1023) >> at org.mortbay.jetty.Response.sendError(Response.java:240) >> at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:3627) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:707) >> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) >> at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) >> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221) >> at org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:816) >> at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212) >> >> What could be the reason behind it ? The job should have finished in 10 min. but it's going on and on for hours. >> >> >> Regards, >> Ajay Srivastava >> >> >> > > > > -- > Harsh J |