Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Re: why my terasort job become a local job?


+
Jeff Zhang 2013-11-26, 05:36
Copy link to this message
-
Re: why my terasort job become a local job?
ch huang 2013-11-26, 05:55
yes ,i did

# grep -C 3 framework /etc/hadoop/conf/mapred-site.xml
<configuration>
<!-- YARN -->
    <property>
        <name>mapreduce.framework.name</name>
        <value>yarn</value>
    </property>

On Tue, Nov 26, 2013 at 1:36 PM, Jeff Zhang <[EMAIL PROTECTED]> wrote:

> Do you set to use yarn framework in mapred-site.xml as following ?
>
>    <property>
>    <name>mapreduce.framework.name</name>
>    <value>yarn</value>
>   </property>
>
>
> On Tue, Nov 26, 2013 at 1:27 PM, ch huang <[EMAIL PROTECTED]> wrote:
>
>> hi,maillist:
>>           i run terasort in my hadoop cluster,and it run as a local job,i
>> do not know why ,anyone can help?
>>
>> i use hadoop version is CDH4.4
>>
>> # sudo -u hdfs hadoop jar
>> /usr/lib/hadoop-mapreduce/hadoop-mapreduce-examples-2.0.0-cdh4.4.0.jar
>> teragen 10000000 /alex/terasort/10G-input
>> ......
>>
>> 13/11/26 11:57:39 INFO mapred.Task:
>> Task:attempt_local321416814_0001_m_000000_0 is done. And is in the process
>> of commiting
>> 13/11/26 11:57:39 INFO mapred.LocalJobRunner:
>> 13/11/26 11:57:39 INFO mapred.Task: Task
>> attempt_local321416814_0001_m_000000_0 is allowed to commit now
>> 13/11/26 11:57:39 INFO mapred.FileOutputCommitter: Saved output of task
>> 'attempt_local321416814_0001_m_000000_0' to
>> hdfs://product/alex/terasort/10G-input
>> 13/11/26 11:57:39 INFO mapred.LocalJobRunner:
>> 13/11/26 11:57:39 INFO mapred.Task: Task
>> 'attempt_local321416814_0001_m_000000_0' done.
>> 13/11/26 11:57:39 INFO mapred.LocalJobRunner: Finishing task:
>> attempt_local321416814_0001_m_000000_0
>> 13/11/26 11:57:39 INFO mapred.LocalJobRunner: Map task executor complete.
>> 13/11/26 11:57:40 INFO mapred.JobClient:  map 100% reduce 0%
>> 13/11/26 11:57:40 INFO mapred.JobClient: Job complete:
>> job_local321416814_0001
>> 13/11/26 11:57:40 INFO mapred.JobClient: Counters: 19
>>
>
>