Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: _temporary doesn't exist


Copy link to this message
-
Re: _temporary doesn't exist
Hi Vipul
      AFAIK, the clean up should happen after the job completion not task.
But what is causing this clean up, may be you can get some info from the NN
logs.

Regards
Bejoy

On Sat, Mar 17, 2012 at 12:48 AM, Vipul Bharakhada <[EMAIL PROTECTED]>wrote:

> Its one of the server which I don't have permission to upgrade. but any
> help is fine. I saw a bug filed on 0.20 that mapreduce doesnt check for the
> _temporary directory existence. but not so sure is it hadoop or its one of
> the scheduled jobs on server which is cleaning up hadoop. I am curious
> about why its deleting the _temporary folder before the task is finished.
> the clean up should happen at the task completion If I am not wrong.
> Correct me If I am wrong. I am new to hadoop.
> Thank you.
> -Vipul
>
> On Fri, Mar 16, 2012 at 12:09 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
>
> > Hi Vipul
> >      Is there any reason you are with 0.17 version of hadoop? It is a
> > pretty old version of hadoop (more than 2 years back) and tons of bug
> fixes
> > and optimizations have gone into trunk after the same. You should badly
> > upgrade to any 1.0.X releases. It would be hard for any one on the list
> to
> > help you out with such an outdated version,Try an upgrade and
> > see whether this issue still persists.
> >
> > Regards
> > Bejoy
> >
> > On Sat, Mar 17, 2012 at 12:27 AM, Vipul Bharakhada <[EMAIL PROTECTED]
> > >wrote:
> >
> > > One more observation: usually this job takes 3 to 4 minutes, however
> when
> > > it fails, at that particular time it takes more than 42 to 50 minutes.
> > > -Vipul
> > >
> > > On Fri, Mar 16, 2012 at 11:38 AM, Vipul Bharakhada <
> [EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Hi,
> > > > I am using the old hadoop version 0.17.2 and I am getting the
> following
> > > > exception when I am trying to run a Job. But it only happens at
> > > particular
> > > > time, as cron jobs do run those task at particular intervals but it
> > only
> > > > fails at one particular time in day.
> > > >
> > > > Mar 14 06:49:23 7 08884: java.io.IOException: The directory
> > > > hdfs://<{IPADDRESS}>:<{PORT}>/myserver/matcher/output/_temporary
> doesnt
> > > > exist
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.localizeTask(TaskTracker.java:1439)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker$TaskInProgress.launchTask(TaskTracker.java:1511)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> > >
> >
> org.apache.hadoop.mapred.TaskTracker.launchTaskForJob(TaskTracker.java:723)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> org.apache.hadoop.mapred.TaskTracker.localizeJob(TaskTracker.java:716)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> > org.apache.hadoop.mapred.TaskTracker.startNewTask(TaskTracker.java:1274)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > >
> org.apache.hadoop.mapred.TaskTracker.offerService(TaskTracker.java:915)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > > org.apache.hadoop.mapred.TaskTracker.run(TaskTracker.java:1310)
> > > > Mar 14 06:49:23 7 08884: \09at
> > > > org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:2251)
> > > >
> > > > What can be a problem? as this folder is created by hadoop internally
> > and
> > > > used internally, clean up is also done internally by hadoop. So why
> > this
> > > > directory is missing at some particular time?
> > > > Any clue?
> > > > -Vipul
> > > >
> > > >
> > >
> >
>