Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> My Job always in 'SUBMIT' state. Stuck at Map: 0% - Reduce 0%


Copy link to this message
-
Re: My Job always in 'SUBMIT' state. Stuck at Map: 0% - Reduce 0%
Could you debug and check the Job Tracker/your scheduler code, then you can
pinpoint the exact issue and fix it.

Thanks
Devaraj k
On Thu, Jul 11, 2013 at 7:58 PM, Sreejith Ramakrishnan <
[EMAIL PROTECTED]> wrote:

> @Devaraj K
>
> I tried setting mapred.jobtracker.maxtasks.per.job to 4 in mapred-site.xml.
> In the subsequent job, I got this in the log which clearly shows the
> configuration has taken effect:
>
> INFO org.apache.hadoop.mapred.JobInProgress: job_201307111721_0001: nMaps=1
> nReduces=1 max=4
>
> Yet, the original problem remains. It doesn't go anywhere from 0%. Btw,
> when we get the jobQueue, should we specify a name for the queue we want?
> (eg. default)
>
> INFO org.apache.hadoop.mapred.JobTracker: Job job_201307111721_0001 added
> successfully for user 'sreejith' to queue 'default'
>
> Also, is there a limit to the no.of tasks which can be obtained in a
> heartbeat?
>
>
> On Thu, Jul 11, 2013 at 2:15 PM, Devaraj k <[EMAIL PROTECTED]> wrote:
>
> > You don't need to explicitly call the initTasks(), you can find this
> > invocation in JobTracker.java class.
> >
> > >*Yet the **Cannot create task split for <my-job-id>** is still there*.
> > >About errors relating to the JobID, is this significant? --- INFO
> > >org.apache.hadoop.mapred.JobInProgress: job_201307111248_0002: nMaps=1
> > >nReduces=1 max=-1
> >
> > Here it shows max=-1, which means max configured(i.e default) tasks for
> > the Job is -1.
> >
> > I think you can see this exception from JT log.
> >
> > LOG.info(jobId + ": nMaps=" + numMapTasks + " nReduces=" + numReduceTasks
> > + " max=" + maxTasks);
> >     if (maxTasks > 0 && (numMapTasks + numReduceTasks) > maxTasks) {
> >       throw new IOException(
> >                 "The number of tasks for this job " +
> >                 (numMapTasks + numReduceTasks) +
> >                 " exceeds the configured limit " + maxTasks);
> >     }
> >
> > Can you try configuring some number for this configuration
> > 'mapred.jobtracker.maxtasks.per.job' and run the same Job?
> >
> >
> > Thanks
> > Devaraj k
> >
> >
> > -----Original Message-----
> > From: Sreejith Ramakrishnan [mailto:[EMAIL PROTECTED]]
> > Sent: 11 July 2013 13:26
> > To: [EMAIL PROTECTED]
> > Subject: Re: My Job always in 'SUBMIT' state. Stuck at Map: 0% - Reduce
> 0%
> >
> > @Devaraj K
> >
> > *I tried explicitly calling incomingJob.initTasks()*and it gave me a
> > pleasant INFO org.apache.hadoop.mapred.JobInProgress: Job
> > job_201307111248_0002 initialized successfully with 1 map tasks and 1
> > reduce tasks
> >
> > *Yet the **Cannot create task split for <my-job-id>** is still there*.
> > About errors relating to the JobID, is this significant? --- INFO
> > org.apache.hadoop.mapred.JobInProgress: job_201307111248_0002: nMaps=1
> > nReduces=1 max=-1
> >
> > *P.S: In the log file I shared, anything with "sreejith == " were logged
> > by me. Not system-generated*
> >
> >
> > On Thu, Jul 11, 2013 at 1:18 PM, Sreejith Ramakrishnan <
> > [EMAIL PROTECTED]> wrote:
> >
> > > I've attached the relevant portion of the log.
> > >
> > > For easy testing, *I've attached the maven project* for the
> > > EDFJobScheduler. Just make a JAR from it and put it in
> > > $HADOOP_HOME/lib and add this property to *mapred-site.xml*:
> > >
> > > *<property>
> > > *
> > > *    <name>mapred.jobtracker.taskScheduler</name>
> > >     <value>org.apache.hadoop.mapred.EDFJobScheduler</value>
> > > *
> > > *</property>*
> > >
> > >
> > > On Thu, Jul 11, 2013 at 12:19 PM, Sreejith Ramakrishnan <
> > > [EMAIL PROTECTED]> wrote:
> > >
> > >> I'm making my own scheduler. Since I had errors, I rewrote the
> > >> scheduler to do just a simple thing. When I give it a job, it
> > >> allocates 1 map() and 1
> > >> reduce() to it. But that isn't happening. I used a barebones WordCount
> > pgm.
> > >> I'm stuck at map: 0% reduce 0%. And in the jobtracker logs, I see:
> > >>
> > >> INFO org.apache.hadoop.mapred.JobInProgress: Cannot create task split
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB