Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Modify the number of map tasks


Copy link to this message
-
Re: Modify the number of map tasks
I wonder how you are able to run the job without a JT. You must have this
on your mapred-site.xml file :
        <property>
<name>mapred.job.tracker</name>
 <value>localhost:9001</value>
</property>

Also add "hadoop.tmp.dir" in core-site.xml, and "dfs.name.dir" &
"dfs.data.dir" in hdfs-site.xml.

Regards,
    Mohammad Tariq

On Wed, Dec 12, 2012 at 6:46 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:

> For mapred-site.xml :
>
> <configuration>
>
> <property>
> <name>mapred.map.tasks</name>
> <value>6</value>
> </property>
>
> </configuration>
>
> for core-site.xml :
> <configuration>
>
> <!-- <property>
> <name>fs.default.name</name>
> <value>hdfs://localhost:9100</value>
> </property> -->
>
> </configuration>
>
>  on hdfs-site.xml  nothing
>
>
>
>
>
> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>
>> Can I have a look at your config files?
>>
>> Regards,
>>     Mohammad Tariq
>>
>>
>>
>> On Wed, Dec 12, 2012 at 6:31 PM, imen Megdiche <[EMAIL PROTECTED]>wrote:
>>
>>> i run the start-all.sh and all daemons starts without problems. But i
>>> the log of the tasktracker look like this :
>>>
>>>
>>> 2012-12-12 13:53:45,495 INFO org.apache.hadoop.mapred.TaskTracker:
>>> STARTUP_MSG:
>>> /************************************************************
>>> STARTUP_MSG: Starting TaskTracker
>>> STARTUP_MSG:   host = megdiche-OptiPlex-GX280/127.0.1.1
>>> STARTUP_MSG:   args = []
>>> STARTUP_MSG:   version = 1.0.4
>>> STARTUP_MSG:   build >>> https://svn.apache.org/repos/asf/hadoop/common/branches/branch-1.0 -r
>>> 1393290; compiled by 'hortonfo' on Wed Oct  3 05:13:58 UTC 2012
>>> ************************************************************/
>>> 2012-12-12 13:53:47,009 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsConfig: loaded properties from
>>> hadoop-metrics2.properties
>>> 2012-12-12 13:53:47,331 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source
>>> MetricsSystem,sub=Stats registered.
>>> 2012-12-12 13:53:47,336 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Scheduled snapshot
>>> period at 10 second(s).
>>> 2012-12-12 13:53:47,336 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: TaskTracker metrics
>>> system started
>>> 2012-12-12 13:53:48,165 INFO
>>> org.apache.hadoop.metrics2.impl.MetricsSourceAdapter: MBean for source ugi
>>> registered.
>>> 2012-12-12 13:53:48,192 WARN
>>> org.apache.hadoop.metrics2.impl.MetricsSystemImpl: Source name ugi already
>>> exists!
>>> 2012-12-12 13:53:48,513 ERROR org.apache.hadoop.mapred.TaskTracker: Can
>>> not start task tracker because java.lang.IllegalArgumentException: Does not
>>> contain a valid host:port authority: local
>>>     at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:162)
>>>     at org.apache.hadoop.net.NetUtils.createSocketAddr(NetUtils.java:128)
>>>     at
>>> org.apache.hadoop.mapred.JobTracker.getAddress(JobTracker.java:2560)
>>>     at org.apache.hadoop.mapred.TaskTracker.<init>(TaskTracker.java:1426)
>>>     at org.apache.hadoop.mapred.TaskTracker.main(TaskTracker.java:3742)
>>>
>>> 2012-12-12 13:53:48,519 INFO org.apache.hadoop.mapred.TaskTracker:
>>> SHUTDOWN_MSG:
>>> /************************************************************
>>> SHUTDOWN_MSG: Shutting down TaskTracker at megdiche-OptiPlex-GX280/
>>> 127.0.1.1
>>> ************************************************************/
>>>
>>>
>>>
>>>
>>> 2012/12/12 Mohammad Tariq <[EMAIL PROTECTED]>
>>>
>>>> I would check if all the daemons are running properly or not, before
>>>> anything else. If some problem is found, next place to track is the log of
>>>> each daemon.
>>>>
>>>> The correct command to check the status of a job from command line is :
>>>> hadoop job -status jobID.
>>>> (Mind the 'space' after job and remove 'command' from the statement)
>>>>
>>>> HTH
>>>>
>>>> Regards,
>>>>     Mohammad Tariq
>>>>
>>>>
>>>>
>>>> On Wed, Dec 12, 2012 at 6:14 PM, imen Megdiche <[EMAIL PROTECTED]
>>>> > wrote:
>>>>
>>>>> My goal is to analyze the response time of MapReduce depending on the size
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB