Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Modify the number of map tasks


+
imen Megdiche 2012-12-12, 10:41
+
Mohammad Tariq 2012-12-12, 11:04
+
imen Megdiche 2012-12-12, 11:41
+
Mohammad Tariq 2012-12-12, 12:06
+
imen Megdiche 2012-12-12, 12:11
+
imen Megdiche 2012-12-12, 12:12
+
Mohammad Tariq 2012-12-12, 12:19
+
imen Megdiche 2012-12-12, 12:23
+
Mohammad Tariq 2012-12-12, 12:25
+
imen Megdiche 2012-12-12, 12:30
+
Mohammad Tariq 2012-12-12, 12:36
+
imen Megdiche 2012-12-12, 12:44
Copy link to this message
-
Re: Modify the number of map tasks
Mohammad Tariq 2012-12-12, 12:53
I would check if all the daemons are running properly or not, before
anything else. If some problem is found, next place to track is the log of
each daemon.

The correct command to check the status of a job from command line is :
hadoop job -status jobID.
(Mind the 'space' after job and remove 'command' from the statement)

HTH

Regards,
    Mohammad Tariq

On Wed, Dec 12, 2012 at 6:14 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:

> My goal is to analyze the response time of MapReduce depending on the size
> of the input files. I need to change the number of map and / or Reduce
> tasks and recover the execution time. S it turns out that nothing works locally
> on my pc :
> neither hadoop job-status command job_local_0001 (which return no job
> found )
> nor localhost: 50030
> I will be very grateful if you can help m better understand these problem
>
>
> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>
>> Are you working locally?What exactly is the issue?
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>>
>> On Wed, Dec 12, 2012 at 6:00 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>>
>>> no
>>>
>>>
>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>
>>>> Any luck with "localhost:50030"??
>>>>
>>>> Regards,
>>>>     Mohammad Tariq
>>>>
>>>>
>>>>
>>>> On Wed, Dec 12, 2012 at 5:53 PM, imen Megdiche <[EMAIL PROTECTED]
>>>> > wrote:
>>>>
>>>>> i run the job through the command line
>>>>>
>>>>>
>>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>>
>>>>>> You have to replace "JobTrackerHost" in "JobTrackerHost:50030" with
>>>>>> the actual name of the machine where JobTracker is running. For
>>>>>> example, If you are working on a local cluster, you have to use
>>>>>> "localhost:50030".
>>>>>>
>>>>>> Are you running your job through the command line or some IDE?
>>>>>>
>>>>>> Regards,
>>>>>>     Mohammad Tariq
>>>>>>
>>>>>>
>>>>>>
>>>>>> On Wed, Dec 12, 2012 at 5:42 PM, imen Megdiche <
>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>
>>>>>>> excuse me the data size is 98 MB
>>>>>>>
>>>>>>>
>>>>>>> 2012/12/12 imen Megdiche <[EMAIL PROTECTED]>
>>>>>>>
>>>>>>>> the size of data 49 MB and n of map 4
>>>>>>>> the web UI JobTrackerHost:50030 does not wok, what should i do to
>>>>>>>> make this appear , i work on ubuntu
>>>>>>>>
>>>>>>>>
>>>>>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>>>>>
>>>>>>>>> Hi Imen,
>>>>>>>>>
>>>>>>>>>      You can visit the MR web UI at "JobTrackerHost:50030" and see
>>>>>>>>> all the useful information like no. of mappers, no of reducers, time taken
>>>>>>>>>  for the execution etc.
>>>>>>>>>
>>>>>>>>> One quick question for you, what is the size of your data and what
>>>>>>>>> is the no of maps which you are getting right now?
>>>>>>>>>
>>>>>>>>> Regards,
>>>>>>>>>     Mohammad Tariq
>>>>>>>>>
>>>>>>>>>
>>>>>>>>>
>>>>>>>>> On Wed, Dec 12, 2012 at 5:11 PM, imen Megdiche <
>>>>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>>>>
>>>>>>>>>> Thank you Mohammad but the number of map tasks still the same in
>>>>>>>>>> the execution. Do you know how to capture the time spent on execution.
>>>>>>>>>>
>>>>>>>>>>
>>>>>>>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>>>>>>>
>>>>>>>>>>> Hi Imen,
>>>>>>>>>>>
>>>>>>>>>>>     You can add "mapred.map.tasks" property in your
>>>>>>>>>>> mapred-site.xml file.
>>>>>>>>>>>
>>>>>>>>>>> But, it is just a hint for the InputFormat. Actually no. of maps
>>>>>>>>>>> is actually determined by the no of InputSplits created by the InputFormat.
>>>>>>>>>>>
>>>>>>>>>>> HTH
>>>>>>>>>>>
>>>>>>>>>>> Regards,
>>>>>>>>>>>     Mohammad Tariq
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>>> On Wed, Dec 12, 2012 at 4:11 PM, imen Megdiche <
>>>>>>>>>>> [EMAIL PROTECTED]> wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi,
>>>>>>>>>>>>
>>>>>>>>>>>> I try to force the number of map for the mapreduce job with the
>>>>>>>>>>>> command :
>>>>>>>>>>>>   public static void main(String[] args) throws Exception {
>>>>>>>>>>>>
>>>>>>>>>>>>           JobConf conf = new JobConf(WordCount.class);
+
imen Megdiche 2012-12-12, 13:01
+
Mohammad Tariq 2012-12-12, 13:07
+
imen Megdiche 2012-12-12, 13:16
+
Mohammad Tariq 2012-12-12, 13:22
+
imen Megdiche 2012-12-12, 13:38
+
Mohammad Tariq 2012-12-12, 13:48
+
imen Megdiche 2012-12-12, 14:01
+
Mohammad Tariq 2012-12-12, 14:07
+
imen Megdiche 2012-12-12, 14:15
+
Mohammad Tariq 2012-12-12, 14:48
+
imen Megdiche 2012-12-12, 15:25
+
Mohammad Tariq 2012-12-12, 15:37