-Re: how to restrict the concurrent running map tasks?
Harsh J 2013-01-18, 11:44
You will need to use an alternative scheduler for this.
Look at minMaps/maxMaps/etc. properties in FairScheduler at
Alternatively, look at resource-based scheduling in CapacityScheduler at
P.s. Do not use general@ list for user level queries. The right list is
On Fri, Jan 18, 2013 at 3:52 PM, hwang <[EMAIL PROTECTED]> wrote:
> Hi all:
> My hadoop version is 1.0.2. Now I want at most 10 map tasks running at the
> same time. I have found 2 parameter related to this question.
> a) mapred.job.map.capacity
> but in my hadoop version, this parameter seems abandoned.
> b) mapred.jobtracker.taskScheduler.maxRunningTasksPerJob (
> I set this variable like below:
> Configuration conf = new Configuration();
> conf.set("date", date);
> conf.set("mapred.job.queue.name", "hadoop");
> conf.set("mapred.jobtracker.taskScheduler.maxRunningTasksPerJob", "10");
> Job job = new Job(conf, "ConstructApkDownload_" + date);
> The problem is that it doesn't work. There is still more than 50 maps
> running as the job starts.
> I'm not sure whether I set this parameter in wrong way ? or misunderstand
> After looking through the hadoop document, I can't find another parameter
> to limit the concurrent running map tasks.
> Hope someone can help me ,Thanks.