Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Jobs randomly not starting


Copy link to this message
-
Jobs randomly not starting
Robert Dyer 2012-07-13, 04:03
I'm using Hadoop 1.0.3 on a small cluster (1 namenode, 1 jobtracker, 2
compute nodes).  My input size is a sequence file of around 280mb.

Generally, my jobs run just fine and all finish in 2-5 minutes.  However,
quite randomly the jobs refuse to run.  They submit and appear when running
'hadoop job -list' but don't appear on the jobtracker's webpage.  If I
manually type in the job ID on the webpage I can see it is trying to run
the setup task - the map tasks haven't even started.  I've left them to run
and even after several minutes it is still in this state.

When I spot this, I kill the job and resubmit it and generally it works.

A couple of times I have seen similar problems with reduce tasks that get
stuck while 'initializing'.

Any ideas?
+
Harsh J 2012-07-13, 06:04
+
Robert Dyer 2012-07-17, 20:27
+
Bejoy KS 2012-07-13, 04:38