Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive >> mail # user >> Modify the number of map tasks


+
imen Megdiche 2012-12-12, 10:41
+
Mohammad Tariq 2012-12-12, 11:04
+
imen Megdiche 2012-12-12, 11:41
+
Mohammad Tariq 2012-12-12, 12:06
+
imen Megdiche 2012-12-12, 12:11
+
imen Megdiche 2012-12-12, 12:12
+
Mohammad Tariq 2012-12-12, 12:19
+
imen Megdiche 2012-12-12, 12:23
+
Mohammad Tariq 2012-12-12, 12:25
+
imen Megdiche 2012-12-12, 12:30
+
Mohammad Tariq 2012-12-12, 12:36
+
imen Megdiche 2012-12-12, 12:44
+
Mohammad Tariq 2012-12-12, 12:53
Copy link to this message
-
Re: Modify the number of map tasks
i run the start-all.sh and all daemons starts without problems. But i the
log of the tasktracker look like this :
2012-12-12 13:53:45,495 INFO org.apache.hadoop.mapred.TaskTracker:
STARTUP_MSG:
/************************************************************
STARTUP_MSG: Starting TaskTracker
STARTUP_MSG:   host = megdiche-OptiPlex-GX280/127.0.1.1
STARTUP_MSG:   args = []
STARTUP_MSG:   version = 1.0.4
STARTUP_MSG:   build https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
1393290; compiled by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
************************************************************/
2012-12-12 13:53:47,009 INFO org.apache.hadoop.metrics2.impl.MetricsConfig:
loaded properties from hadoop-metrics2.properties
2012-12-12 13:53:47,331 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
MetricsSystem,sub=Stats registered.
2012-12-12 13:53:47,336 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
period at 10 second(s).
2012-12-12 13:53:47,336 INFO
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics
system started
2012-12-12 13:53:48,165 INFO
org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
registered.
2012-12-12 13:53:48,192 WARN
org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
exists!
2012-12-12 13:53:48,513 ERROR org.apache.hadoop.mapred.TaskTracker: Can not
start task tracker because java.lang.IllegalArgumentException: Does not
contain a valid host:port authority: local
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:162)
    at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:128)
    at org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2560)
    at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1426)
    at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)

2012-12-12 13:53:48,519 INFO org.apache.hadoop.mapred.TaskTracker:
SHUTDOWN_MSG:
/************************************************************
SHUTDOWN_MSG: Shutting down TaskTracker at megdiche-OptiPlex-GX280/127.0.1.1
************************************************************/
2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>

> I would check if all the daemons are running properly or not, before
> anything else. If some problem is found, next place to track is the log of
> each daemon.
>
> The correct command to check the status of a job from command line is :
> hadoop job -status jobID.
> (Mind the 'space' after job and remove 'command' from the statement)
>
> HTH
>
> Regards,
>     Mohammad Tariq
>
>
>
> On Wed, Dec 12, 2012 at 6:14 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>
>> My goal is to analyze the response time of MapReduce depending on the size
>> of the input files. I need to change the number of map and / or Reduce
>> tasks and recover the execution time. S it turns out that nothing works locally
>> on my pc :
>> neither hadoop job-status command job_local_0001 (which return no job
>> found )
>> nor localhost: 50030
>> I will be very grateful if you can help m better understand these problem
>>
>>
>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>
>>> Are you working locally?What exactly is the issue?
>>>
>>> Regards,
>>>     Mohammad Tariq
>>>
>>>
>>>
>>> On Wed, Dec 12, 2012 at 6:00 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>>>
>>>> no
>>>>
>>>>
>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>
>>>>> Any luck with "localhost:50030"??
>>>>>
>>>>> Regards,
>>>>>     Mohammad Tariq
>>>>>
>>>>>
>>>>>
>>>>> On Wed, Dec 12, 2012 at 5:53 PM, imen Megdiche <
>>>>> [EMAIL PROTECTED]> wrote:
>>>>>
>>>>>> i run the job through the command line
>>>>>>
>>>>>>
>>>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>>>
>>>>>>> You have to replace "JobTrackerHost" in "JobTrackerHost:50030" with
>>>>>>> the actual name of the machine where JobTracker is running. For
>>>>>>> example, If you are working on a local cluster, you have to use
+
Mohammad Tariq 2012-12-12, 13:07
+
imen Megdiche 2012-12-12, 13:16
+
Mohammad Tariq 2012-12-12, 13:22
+
imen Megdiche 2012-12-12, 13:38
+
Mohammad Tariq 2012-12-12, 13:48
+
imen Megdiche 2012-12-12, 14:01
+
Mohammad Tariq 2012-12-12, 14:07
+
imen Megdiche 2012-12-12, 14:15
+
Mohammad Tariq 2012-12-12, 14:48
+
imen Megdiche 2012-12-12, 15:25
+
Mohammad Tariq 2012-12-12, 15:37
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB