Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Modify the number of map tasks


Copy link to this message
-
Re: Modify the number of map tasks
imen Megdiche 2012-12-12, 12:12
excuse me the data size is 98 MB
2012/12/12 imen Megdiche <[EMAIL PROTECTED]>

> the size of data 49 MB and n of map 4
> the web UI JobTrackerHost:50030 does not wok, what should i do to make
> this appear , i work on ubuntu
>
>
> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>
>> Hi Imen,
>>
>>      You can visit the MR web UI at "JobTrackerHost:50030" and see all
>> the useful information like no. of mappers, no of reducers, time taken  for
>> the execution etc.
>>
>> One quick question for you, what is the size of your data and what is the
>> no of maps which you are getting right now?
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>>
>> On Wed, Dec 12, 2012 at 5:11 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>>
>>> Thank you Mohammad but the number of map tasks still the same in the
>>> execution. Do you know how to capture the time spent on execution.
>>>
>>>
>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>
>>>> Hi Imen,
>>>>
>>>>     You can add "mapred.map.tasks" property in your mapred-site.xml
>>>> file.
>>>>
>>>> But, it is just a hint for the InputFormat. Actually no. of maps is
>>>> actually determined by the no of InputSplits created by the InputFormat.
>>>>
>>>> HTH
>>>>
>>>> Regards,
>>>>     Mohammad Tariq
>>>>
>>>>
>>>>
>>>> On Wed, Dec 12, 2012 at 4:11 PM, imen Megdiche <[EMAIL PROTECTED]
>>>> > wrote:
>>>>
>>>>> Hi,
>>>>>
>>>>> I try to force the number of map for the mapreduce job with the
>>>>> command :
>>>>>   public static void main(String[] args) throws Exception {
>>>>>
>>>>>           JobConf conf = new JobConf(WordCount.class);
>>>>>              conf.set("mapred.job.tracker", "local");
>>>>>          conf.set("fs.default.name", "local");
>>>>>           conf.setJobName("wordcount");
>>>>>
>>>>>           conf.setOutputKeyClass(Text.class);
>>>>>          conf.setOutputValueClass(IntWritable.class);
>>>>>
>>>>>            conf.setNumMapTask(6);
>>>>>           conf.setMapperClass(Map.class);
>>>>>           conf.setCombinerClass(Reduce.class);
>>>>>           conf.setReducerClass(Reduce.class);
>>>>> ...
>>>>> }
>>>>>
>>>>> But it doesn t work.
>>>>> What can i do to modify the number of map and reduce tasks.
>>>>>
>>>>> Thank you
>>>>>
>>>>
>>>>
>>>
>>
>