Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Modify the number of map tasks


+
imen Megdiche 2012-12-12, 10:41
+
Mohammad Tariq 2012-12-12, 11:04
+
imen Megdiche 2012-12-12, 11:41
Copy link to this message
-
Re: Modify the number of map tasks
Mohammad Tariq 2012-12-12, 12:06
Hi Imen,

     You can visit the MR web UI at "JobTrackerHost:50030" and see all the
useful information like no. of mappers, no of reducers, time taken  for the
execution etc.

One quick question for you, what is the size of your data and what is the
no of maps which you are getting right now?

Regards,
    Mohammad Tariq

On Wed, Dec 12, 2012 at 5:11 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:

> Thank you Mohammad but the number of map tasks still the same in the
> execution. Do you know how to capture the time spent on execution.
>
>
> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>
>> Hi Imen,
>>
>>     You can add "mapred.map.tasks" property in your mapred-site.xml file.
>>
>> But, it is just a hint for the InputFormat. Actually no. of maps is
>> actually determined by the no of InputSplits created by the InputFormat.
>>
>> HTH
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>>
>> On Wed, Dec 12, 2012 at 4:11 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>>
>>> Hi,
>>>
>>> I try to force the number of map for the mapreduce job with the command
>>> :
>>>   public static void main(String[] args) throws Exception {
>>>
>>>           JobConf conf = new JobConf(WordCount.class);
>>>              conf.set("mapred.job.tracker", "local");
>>>          conf.set("fs.default.name", "local");
>>>           conf.setJobName("wordcount");
>>>
>>>           conf.setOutputKeyClass(Text.class);
>>>          conf.setOutputValueClass(IntWritable.class);
>>>
>>>            conf.setNumMapTask(6);
>>>           conf.setMapperClass(Map.class);
>>>           conf.setCombinerClass(Reduce.class);
>>>           conf.setReducerClass(Reduce.class);
>>> ...
>>> }
>>>
>>> But it doesn t work.
>>> What can i do to modify the number of map and reduce tasks.
>>>
>>> Thank you
>>>
>>
>>
>
+
imen Megdiche 2012-12-12, 12:11
+
imen Megdiche 2012-12-12, 12:12
+
Mohammad Tariq 2012-12-12, 12:19
+
imen Megdiche 2012-12-12, 12:23
+
Mohammad Tariq 2012-12-12, 12:25
+
imen Megdiche 2012-12-12, 12:30
+
Mohammad Tariq 2012-12-12, 12:36
+
imen Megdiche 2012-12-12, 12:44
+
Mohammad Tariq 2012-12-12, 12:53
+
imen Megdiche 2012-12-12, 13:01
+
Mohammad Tariq 2012-12-12, 13:07
+
imen Megdiche 2012-12-12, 13:16
+
Mohammad Tariq 2012-12-12, 13:22
+
imen Megdiche 2012-12-12, 13:38
+
Mohammad Tariq 2012-12-12, 13:48
+
imen Megdiche 2012-12-12, 14:01
+
Mohammad Tariq 2012-12-12, 14:07
+
imen Megdiche 2012-12-12, 14:15
+
Mohammad Tariq 2012-12-12, 14:48
+
imen Megdiche 2012-12-12, 15:25
+
Mohammad Tariq 2012-12-12, 15:37