Hi, all,

 

         I need help.

        

         I setup my cluster use hadoop 2.0.1 alpha.

         In HA mode.

         When I run wordcount example using :

         hadoop jar /usr/lib/hadoop-2.0.1-alpha/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.1-alpha.jar wordcount /tmp/input /tmp/output73

         I got errors below.

 

         Please help.

 

         Thanks.

 

        

        

 

 

 

[hdfs@baby20 ~]$ hadoop jar /usr/lib/hadoop-2.0.1-alpha/share/hadoop/mapreduce/hadoop-mapreduce-examples-2.0.1-alpha.jar wordcount /tmp/input /tmp/output73

12/09/14 20:49:49 INFO input.FileInputFormat: Total input paths to process : 2

12/09/14 20:49:49 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable

12/09/14 20:49:49 WARN snappy.LoadSnappy: Snappy native library not loaded

12/09/14 20:49:50 INFO mapreduce.JobSubmitter: number of splits:2

12/09/14 20:49:50 WARN conf.Configuration: mapred.jar is deprecated. Instead, use mapreduce.job.jar

12/09/14 20:49:50 WARN conf.Configuration: mapred.output.value.class is deprecated. Instead, use mapreduce.job.output.value.class

12/09/14 20:49:50 WARN conf.Configuration: mapreduce.combine.class is deprecated. Instead, use mapreduce.job.combine.class

12/09/14 20:49:50 WARN conf.Configuration: mapreduce.map.class is deprecated. Instead, use mapreduce.job.map.class

12/09/14 20:49:50 WARN conf.Configuration: mapred.job.name is deprecated. Instead, use mapreduce.job.name

12/09/14 20:49:50 WARN conf.Configuration: mapreduce.reduce.class is deprecated. Instead, use mapreduce.job.reduce.class

12/09/14 20:49:50 WARN conf.Configuration: mapred.input.dir is deprecated. Instead, use mapreduce.input.fileinputformat.inputdir

12/09/14 20:49:50 WARN conf.Configuration: mapred.output.dir is deprecated. Instead, use mapreduce.output.fileoutputformat.outputdir

12/09/14 20:49:50 WARN conf.Configuration: mapred.map.tasks is deprecated. Instead, use mapreduce.job.maps

12/09/14 20:49:50 WARN conf.Configuration: mapred.output.key.class is deprecated. Instead, use mapreduce.job.output.key.class

12/09/14 20:49:50 WARN conf.Configuration: mapred.working.dir is deprecated. Instead, use mapreduce.job.working.dir

12/09/14 20:49:51 INFO mapred.ResourceMgrDelegate: Submitted application application_1347626975369_0001 to ResourceManager at baby20/10.1.1.40:8040

12/09/14 20:49:51 INFO mapreduce.Job: The url to track the job: http://baby20:8088/proxy/application_1347626975369_0001/

12/09/14 20:49:51 INFO mapreduce.Job: Running job: job_1347626975369_0001

12/09/14 20:50:05 INFO mapreduce.Job: Job job_1347626975369_0001 running in uber mode : false

12/09/14 20:50:05 INFO mapreduce.Job:  map 0% reduce 0%

12/09/14 20:50:20 INFO mapreduce.Job: Task Id : attempt_1347626975369_0001_m_000001_0, Status : FAILED

Error: Java heap space

12/09/14 20:50:20 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby18:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000001_0&filter=stdout

12/09/14 20:50:20 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby18:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000001_0&filter=stderr

12/09/14 20:50:20 INFO mapreduce.Job: Task Id : attempt_1347626975369_0001_m_000000_0, Status : FAILED

Error: Java heap space

12/09/14 20:50:20 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby18:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000000_0&filter=stdout

12/09/14 20:50:20 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby18:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000000_0&filter=stderr

12/09/14 20:50:30 INFO mapreduce.Job: Task Id : attempt_1347626975369_0001_m_000000_1, Status : FAILED

Error: Java heap space

12/09/14 20:50:30 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby16:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000000_1&filter=stdout

12/09/14 20:50:30 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby16:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000000_1&filter=stderr

12/09/14 20:50:31 INFO mapreduce.Job: Task Id : attempt_1347626975369_0001_m_000001_1, Status : FAILED

Error: Java heap space

12/09/14 20:50:31 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby16:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000001_1&filter=stdout

12/09/14 20:50:31 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby16:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000001_1&filter=stderr

12/09/14 20:50:39 INFO mapreduce.Job: Task Id : attempt_1347626975369_0001_m_000000_2, Status : FAILED

Error: Java heap space

12/09/14 20:50:39 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby16:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000000_2&filter=stdout

12/09/14 20:50:39 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby16:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000000_2&filter=stderr

12/09/14 20:50:39 INFO mapreduce.Job: Task Id : attempt_1347626975369_0001_m_000001_2, Status : FAILED

Error: Java heap space

12/09/14 20:50:39 WARN mapreduce.Job: Error reading task output Server returned HTTP response code: 400 for URL: http://baby16:8080/tasklog?plaintext=true&attemptid=attempt_1347626975369_0001_m_000001_2&filter=stdout

12/09/14 20:50:39 WARN mapreduce.Job: Error reading task outp

The console output as well as the AM logs show that the TaskAttempts are failing because they are out of heap-space. See your configuration below, you should bump them up.

HTH,

+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Sep 14, 2012, at 6:15 AM, Tao wrote:

>         <!-- Larger heap-size for child jvms of maps  -->
>         <property>
>                <name>mapreduce.map.java.opts</name>
>                 <value>-Xmx128m</value>
>         </property>
>  
>         <!-- Larger resource limit for reduces  -->
>         <property>
>                 <name>mapreduce.reduce.memory.mb</name>
>                 <value>512</value>
>         </property>
>  
>         <!-- Larger heap-size for child jvms of reduces  -->
>         <property>
>                 <name>mapreduce.reduce.java.opts</name>
>                 <value>-Xmx256m</value>
>         </property>