Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - Modify the number of map tasks


Copy link to this message
-
Re: Modify the number of map tasks
imen Megdiche 2012-12-12, 13:16
For mapred-site.xml :

<configuration>

<property>
<name>mapred.map.tasks</name>
<value>6</value>
</property>

</configuration>

for core-site.xml :
<configuration>

<!-- <property>
<name>fs.default.name</name>
<value>hdfs://localhost:9100</value>
</property> -->

</configuration>

 on hdfs-site.xml  nothing

2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>

> Can I have a look at your config files?
>
> Regards,
>     Mohammad Tariq
>
>
>
> On Wed, Dec 12, 2012 at 6:31 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>
>> i run the start-all.sh and all daemons starts without problems. But i the
>> log of the tasktracker look like this :
>>
>>
>> 2012-12-12 13:53:45,495 INFO org.apache.hadoop.mapred.TaskTracker:
>> STARTUP_MSG:
>> /************************************************************
>> STARTUP_MSG: Starting TaskTracker
>> STARTUP_MSG:   host = megdiche-OptiPlex-GX280/127.0.1.1
>> STARTUP_MSG:   args = []
>> STARTUP_MSG:   version = 1.0.4
>> STARTUP_MSG:   build >> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
>> 1393290; compiled by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
>> ************************************************************/
>> 2012-12-12 13:53:47,009 INFO
>> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
>> hadoop-metrics2.properties
>> 2012-12-12 13:53:47,331 INFO
>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>> MetricsSystem,sub=Stats registered.
>> 2012-12-12 13:53:47,336 INFO
>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
>> period at 10 second(s).
>> 2012-12-12 13:53:47,336 INFO
>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics
>> system started
>> 2012-12-12 13:53:48,165 INFO
>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
>> registered.
>> 2012-12-12 13:53:48,192 WARN
>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
>> exists!
>> 2012-12-12 13:53:48,513 ERROR org.apache.hadoop.mapred.TaskTracker: Can
>> not start task tracker because java.lang.IllegalArgumentException: Does not
>> contain a valid host:port authority: local
>>     at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:162)
>>     at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:128)
>>     at
>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2560)
>>     at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1426)
>>     at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)
>>
>> 2012-12-12 13:53:48,519 INFO org.apache.hadoop.mapred.TaskTracker:
>> SHUTDOWN_MSG:
>> /************************************************************
>> SHUTDOWN_MSG: Shutting down TaskTracker at megdiche-OptiPlex-GX280/
>> 127.0.1.1
>> ************************************************************/
>>
>>
>>
>>
>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>
>>> I would check if all the daemons are running properly or not, before
>>> anything else. If some problem is found, next place to track is the log of
>>> each daemon.
>>>
>>> The correct command to check the status of a job from command line is :
>>> hadoop job -status jobID.
>>> (Mind the 'space' after job and remove 'command' from the statement)
>>>
>>> HTH
>>>
>>> Regards,
>>>     Mohammad Tariq
>>>
>>>
>>>
>>> On Wed, Dec 12, 2012 at 6:14 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>>>
>>>> My goal is to analyze the response time of MapReduce depending on the size
>>>> of the input files. I need to change the number of map and / or Reduce
>>>> tasks and recover the execution time. S it turns out that nothing works locally
>>>> on my pc :
>>>> neither hadoop job-status command job_local_0001 (which return no job
>>>> found )
>>>> nor localhost: 50030
>>>> I will be very grateful if you can help m better understand these
>>>> problem
>>>>
>>>>
>>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>>
>>>>> Are you working locally?What exactly is the issue?