I have a windows hadoop cluster consists of 8 slaves 1 master node. My hadoop program is a collection of recursive jobs. I create 14 map, 14 reduce tasks in each job. My files are up to 10mb.
My problem is that all jobs are waiting at the end of job. Map %100 Reduce %100 seen on command prompt but it does not progress for 5 minutes. Then it goes to next job execution.
So anyone has an idea?
Thanks.
-
Re: wait at the end of job
Radim Kolar 2012-10-17, 11:58
it tracks data read into buffer, not processed data. sitting at 100 percent is okay
All projects made searchable here are trademarks of the Apache Software Foundation.
Service operated by Sematext