Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Modify the number of map tasks


+
imen Megdiche 2012-12-12, 10:41
+
Mohammad Tariq 2012-12-12, 11:04
+
imen Megdiche 2012-12-12, 11:41
+
Mohammad Tariq 2012-12-12, 12:06
+
imen Megdiche 2012-12-12, 12:11
+
imen Megdiche 2012-12-12, 12:12
+
Mohammad Tariq 2012-12-12, 12:19
+
imen Megdiche 2012-12-12, 12:23
+
Mohammad Tariq 2012-12-12, 12:25
+
imen Megdiche 2012-12-12, 12:30
Copy link to this message
-
Re: Modify the number of map tasks
Are you working locally?What exactly is the issue?

Regards,
    Mohammad Tariq

On Wed, Dec 12, 2012 at 6:00 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:

> no
>
>
> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>
>> Any luck with "localhost:50030"??
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>>
>> On Wed, Dec 12, 2012 at 5:53 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>>
>>> i run the job through the command line
>>>
>>>
>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>
>>>> You have to replace "JobTrackerHost" in "JobTrackerHost:50030" with
>>>> the actual name of the machine where JobTracker is running. For
>>>> example, If you are working on a local cluster, you have to use
>>>> "localhost:50030".
>>>>
>>>> Are you running your job through the command line or some IDE?
>>>>
>>>> Regards,
>>>>     Mohammad Tariq
>>>>
>>>>
>>>>
>>>> On Wed, Dec 12, 2012 at 5:42 PM, imen Megdiche <[EMAIL PROTECTED]
>>>> > wrote:
>>>>
>>>>> excuse me the data size is 98 MB
>>>>>
>>>>>
>>>>> 2012/12/12 imen Megdiche <[EMAIL PROTECTED]>
>>>>>
>>>>>> the size of data 49 MB and n of map 4
>>>>>> the web UI JobTrackerHost:50030 does not wok, what should i do to
>>>>>> make this appear , i work on ubuntu
>>>>>>
>>>>>>
>>>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>>>
>>>>>>> Hi Imen,
>>>>>>>
>>>>>>>      You can visit the MR web UI at "JobTrackerHost:50030" and see
>>>>>>> all the useful information like no. of mappers, no of reducers, time taken
>>>>>>>  for the execution etc.
>>>>>>>
>>>>>>> One quick question for you, what is the size of your data and what
>>>>>>> is the no of maps which you are getting right now?
>>>>>>>
>>>>>>> Regards,
>>>>>>>     Mohammad Tariq
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> On Wed, Dec 12, 2012 at 5:11 PM, imen Megdiche <
>>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>>
>>>>>>>> Thank you Mohammad but the number of map tasks still the same in
>>>>>>>> the execution. Do you know how to capture the time spent on execution.
>>>>>>>>
>>>>>>>>
>>>>>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>>>>>
>>>>>>>>> Hi Imen,
>>>>>>>>>
>>>>>>>>>     You can add "mapred.map.tasks" property in your
>>>>>>>>> mapred-site.xml file.
>>>>>>>>>
>>>>>>>>> But, it is just a hint for the InputFormat. Actually no. of maps
>>>>>>>>> is actually determined by the no of InputSplits created by the InputFormat.
>>>>>>>>>
>>>>>>>>> HTH
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>     Mohammad Tariq
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Dec 12, 2012 at 4:11 PM, imen Megdiche <
>>>>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>>>>
>>>>>>>>>> Hi,
>>>>>>>>>>
>>>>>>>>>> I try to force the number of map for the mapreduce job with the
>>>>>>>>>> command :
>>>>>>>>>>   public static void main(String[] args) throws Exception {
>>>>>>>>>>
>>>>>>>>>>           JobConf conf = new JobConf(WordCount.class);
>>>>>>>>>>              conf.set("mapred.job.tracker", "local");
>>>>>>>>>>          conf.set("fs.default.name", "local");
>>>>>>>>>>           conf.setJobName("wordcount");
>>>>>>>>>>
>>>>>>>>>>           conf.setOutputKeyClass(Text.class);
>>>>>>>>>>          conf.setOutputValueClass(IntWritable.class);
>>>>>>>>>>
>>>>>>>>>>            conf.setNumMapTask(6);
>>>>>>>>>>           conf.setMapperClass(Map.class);
>>>>>>>>>>           conf.setCombinerClass(Reduce.class);
>>>>>>>>>>           conf.setReducerClass(Reduce.class);
>>>>>>>>>> ...
>>>>>>>>>> }
>>>>>>>>>>
>>>>>>>>>> But it doesn t work.
>>>>>>>>>> What can i do to modify the number of map and reduce tasks.
>>>>>>>>>>
>>>>>>>>>> Thank you
>>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>>
>>>
>>
>
+
imen Megdiche 2012-12-12, 12:44
+
Mohammad Tariq 2012-12-12, 12:53
+
imen Megdiche 2012-12-12, 13:01
+
Mohammad Tariq 2012-12-12, 13:07
+
imen Megdiche 2012-12-12, 13:16
+
Mohammad Tariq 2012-12-12, 13:22
+
imen Megdiche 2012-12-12, 13:38
+
Mohammad Tariq 2012-12-12, 13:48
+
imen Megdiche 2012-12-12, 14:01
+
Mohammad Tariq 2012-12-12, 14:07
+
imen Megdiche 2012-12-12, 14:15
+
Mohammad Tariq 2012-12-12, 14:48
+
imen Megdiche 2012-12-12, 15:25
+
Mohammad Tariq 2012-12-12, 15:37