Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - Task failure in slave node


Copy link to this message
-
Re: Task failure in slave node
Azuryy Yu 2013-07-11, 13:40
hi,

put all mahou jars under hadoop_home/lib, then restart cluster.
 On Jul 11, 2013 8:45 PM, "Margusja" <[EMAIL PROTECTED]> wrote:

> Hi
>
> I have tow nodes:
> n1 (master, salve) and n2 (slave)
>
> after set up I ran wordcount example and it worked fine:
> [hduser@n1 ~]$ hadoop jar /usr/local/hadoop/hadoop-**examples-1.0.4.jar
> wordcount /user/hduser/gutenberg /user/hduser/gutenberg-output
> 13/07/11 15:30:44 INFO input.FileInputFormat: Total input paths to process
> : 7
> 13/07/11 15:30:44 INFO util.NativeCodeLoader: Loaded the native-hadoop
> library
> 13/07/11 15:30:44 WARN snappy.LoadSnappy: Snappy native library not loaded
> 13/07/11 15:30:44 INFO mapred.JobClient: Running job: job_201307111355_0015
> 13/07/11 15:30:45 INFO mapred.JobClient:  map 0% reduce 0%
> 13/07/11 15:31:03 INFO mapred.JobClient:  map 42% reduce 0%
> 13/07/11 15:31:06 INFO mapred.JobClient:  map 57% reduce 0%
> 13/07/11 15:31:09 INFO mapred.JobClient:  map 71% reduce 0%
> 13/07/11 15:31:15 INFO mapred.JobClient:  map 100% reduce 0%
> 13/07/11 15:31:18 INFO mapred.JobClient:  map 100% reduce 23%
> 13/07/11 15:31:27 INFO mapred.JobClient:  map 100% reduce 100%
> 13/07/11 15:31:32 INFO mapred.JobClient: Job complete:
> job_201307111355_0015
> 13/07/11 15:31:32 INFO mapred.JobClient: Counters: 30
> 13/07/11 15:31:32 INFO mapred.JobClient:   Job Counters
> 13/07/11 15:31:32 INFO mapred.JobClient:     Launched reduce tasks=1
> 13/07/11 15:31:32 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=67576
> 13/07/11 15:31:32 INFO mapred.JobClient:     Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/07/11 15:31:32 INFO mapred.JobClient:     Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/07/11 15:31:32 INFO mapred.JobClient:     Rack-local map tasks=3
> 13/07/11 15:31:32 INFO mapred.JobClient:     Launched map tasks=7
> 13/07/11 15:31:32 INFO mapred.JobClient:     Data-local map tasks=4
> 13/07/11 15:31:32 INFO mapred.JobClient: SLOTS_MILLIS_REDUCES=21992
> 13/07/11 15:31:32 INFO mapred.JobClient:   File Output Format Counters
> 13/07/11 15:31:32 INFO mapred.JobClient:     Bytes Written=1412505
> 13/07/11 15:31:32 INFO mapred.JobClient:   FileSystemCounters
> 13/07/11 15:31:32 INFO mapred.JobClient:     FILE_BYTES_READ=5414195
> 13/07/11 15:31:32 INFO mapred.JobClient:     HDFS_BYTES_READ=6950820
> 13/07/11 15:31:32 INFO mapred.JobClient: FILE_BYTES_WRITTEN=8744993
> 13/07/11 15:31:32 INFO mapred.JobClient: HDFS_BYTES_WRITTEN=1412505
> 13/07/11 15:31:32 INFO mapred.JobClient:   File Input Format Counters
> 13/07/11 15:31:32 INFO mapred.JobClient:     Bytes Read=6950001
> 13/07/11 15:31:32 INFO mapred.JobClient:   Map-Reduce Framework
> 13/07/11 15:31:32 INFO mapred.JobClient:     Map output materialized
> bytes=3157469
> 13/07/11 15:31:32 INFO mapred.JobClient:     Map input records=137146
> 13/07/11 15:31:32 INFO mapred.JobClient:     Reduce shuffle bytes=2904836
> 13/07/11 15:31:32 INFO mapred.JobClient:     Spilled Records=594764
> 13/07/11 15:31:32 INFO mapred.JobClient:     Map output bytes=11435849
> 13/07/11 15:31:32 INFO mapred.JobClient:     Total committed heap usage
> (bytes)=1128136704
> 13/07/11 15:31:32 INFO mapred.JobClient:     CPU time spent (ms)=18230
> 13/07/11 15:31:32 INFO mapred.JobClient:     Combine input records=1174991
> 13/07/11 15:31:32 INFO mapred.JobClient:     SPLIT_RAW_BYTES=819
> 13/07/11 15:31:32 INFO mapred.JobClient:     Reduce input records=218990
> 13/07/11 15:31:32 INFO mapred.JobClient:     Reduce input groups=128513
> 13/07/11 15:31:32 INFO mapred.JobClient:     Combine output records=218990
> 13/07/11 15:31:32 INFO mapred.JobClient:     Physical memory (bytes)
> snapshot=1179656192
> 13/07/11 15:31:32 INFO mapred.JobClient:     Reduce output records=128513
> 13/07/11 15:31:32 INFO mapred.JobClient:     Virtual memory (bytes)
> snapshot=22992117760
> 13/07/11 15:31:32 INFO mapred.JobClient:     Map output records=1174991
>
> from web interface (http://n1:50030/) I saw that both (n1 and n2 ) were