Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> My Job always in 'SUBMIT' state. Stuck at Map: 0% - Reduce 0%


Copy link to this message
-
RE: My Job always in 'SUBMIT' state. Stuck at Map: 0% - Reduce 0%
It seems tasks initialization is not happening for the Job.  Do you see any other message or error in the JT log related to that Job/Job-Id.

Thanks
Devaraj k

From: Sreejith Ramakrishnan [mailto:[EMAIL PROTECTED]]
Sent: 11 July 2013 12:20
To: [EMAIL PROTECTED]
Subject: My Job always in 'SUBMIT' state. Stuck at Map: 0% - Reduce 0%

I'm making my own scheduler. Since I had errors, I rewrote the scheduler to do just a simple thing. When I give it a job, it allocates 1 map() and 1 reduce() to it. But that isn't happening. I used a barebones WordCount pgm. I'm stuck at map: 0% reduce 0%. And in the jobtracker logs, I see:

INFO org.apache.hadoop.mapred.JobInProgress: Cannot create task split for <my-job-id>
DETAILED INFO:

Here's some snippets of code. The scheduler logic is inside the assignTasks():
We get a JobInProgressListener in the constructor-
this.jobQueueJobInProgressListener = new JobQueueJobInProgressListener();

In start() -
this.taskTrackerManager.addJobInProgressListener(jobQueueJobInProgressListener);

Then I just take the last element in that queue (I assume its the latest submitted job) and assign to incomingJob (of type JobInProgress).

Then I obtain a new map task as follows and append this task to a list which is the return value of the assignTasks():
Task createdMap = incomingJob.obtainNewMapTask(taskTrackerStatus, numTaskTrackers, taskTrackerManager.getNumberOfUniqueHosts());

I've attached the scheduler. Can any expert point out if there's any ignorant mistakes from my side?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB