Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Hadoop 0.20.2: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_201008131730_0001/attempt_201008131730_0001_m_000000_2/output/file.out.index in any of the configured local directories


Copy link to this message
-
Re: Hadoop 0.20.2: org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_201008131730_0001/attempt_201008131730_0001_m_000000_2/output/file.out.index in any of the configured local directories
Aside from usual checks regarding connectivity between master and
slaves, one quick observation is that the hadoop.tmp.dir says 'tmp'.
Shouldn't this be '/tmp' ?

Thanks
Hemanth
2010/8/13 Kevin Chen <[EMAIL PROTECTED]>:
>
> Hello,
>
> I've confused the problem for a week already, Please sharing if you know what  could be causing this, Thinks in advance!
>
> Hadoop version: 0.20.2
>
> Machines:
>
> machine 1 - NameNode,JobTracker,SecondNameNode:
> OS: ubuntu
> Hostname: master
> ulimit -n: 10240
> disk space: enough
> ${mapred.local.dir}'s permissions: 777
>
> machine 2 - DataNode, TaskTracker:
>
> OS: ubuntu
>
> Hostname: slave1
>
> ulimit -n: 10240
> disk space: enough
> ${mapred.local.dir}'s permissions: 777
>
> machine 3 - DataNode, TaskTracker:
>
> OS: ubuntu
>
> Hostname: slave2
>
> ulimit -n: 10240
> disk space: enough
> ${mapred.local.dir}'s permissions: 777
>
>
> Configurations:
>
> core-site.xml:
> <configuration>
>  <property>
>    <name>hadoop.tmp.dir</name>
>    <value>tmp</value>
>  </property>
>  <property>
>    <name>fs.default.name</name>
>    <value>hdfs://master:9000</value>
>  </property>
> </configuration>
>
>
> hdfs-site.xml:
> <configuration>
> </configuration>
>
>
> mapred-site.xml:
> <configuration>
>  <property>
>    <name>mapred.job.tracker</name>
>    <value>master:9001</value>
>  </property>
> </configuration>
>
>
> masters:
> bestgembler@master
>
>
> slaves:
> hadoop@slave1
> hadoop@slave2
>
> Parts of error:
>
>
> Terminal:
> 10/08/13 17:35:03 INFO mapred.FileInputFormat: Total input paths to process : 1
> 10/08/13 17:35:03 INFO mapred.JobClient: Running job: job_201008131730_0001
> 10/08/13 17:35:04 INFO mapred.JobClient:  map 0% reduce 0%
> 10/08/13 17:35:14 INFO mapred.JobClient:  map 50% reduce 0%
> 10/08/13 17:35:17 INFO mapred.JobClient:  map 100% reduce 0%
> 10/08/13 17:35:23 INFO mapred.JobClient: Task Id : attempt_201008131730_0001_m_000001_0, Status : FAILED
> Map output lost, rescheduling: getMapOutput(attempt_201008131730_0001_m_000001_0,0) failed :
> org.apache.hadoop.util.DiskChecker$DiskErrorException: Could not find taskTracker/jobcache/job_201008131730_0001/attempt_201008131730_0001_m_000001_0/output/file.out.index in any of the configured local directories
>    at org.apache.hadoop.fs.LocalDirAllocator$AllocatorPerContext.getLocalPathToRead(LocalDirAllocator.java:389)
>    at org.apache.hadoop.fs.LocalDirAllocator.getLocalPathToRead(LocalDirAllocator.java:138)
>    at org.apache.hadoop.mapred.TaskTracker$MapOutputServlet.doGet(TaskTracker.java:2887)
>    at javax.servlet.http.HttpServlet.service(HttpServlet.java:707)
>    at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
>    at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:502)
>    at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:363)
>    at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
>    at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:181)
>    at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
>    at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:417)
>    at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
>    at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
>    at org.mortbay.jetty.Server.handle(Server.java:324)
>    at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:534)
>    at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:864)
>    at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:533)
>    at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:207)
>    at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:403)
>    at org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:409)
>    at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:522)
>
> hadoop-bestgembler-jobtracker-master.log: