Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Re: error while running TestDFSIO


Copy link to this message
-
Re: error while running TestDFSIO
Ling Kun 2013-04-20, 01:14
I have got the same problem. And also need help.

Ling Kun
On Sat, Apr 20, 2013 at 8:35 AM, kaveh minooie <[EMAIL PROTECTED]> wrote:

> Hi everyone
>
> I am getting this error when i run TestDFSIO. the job actually finishes
> successfully. ( according to jobtracker at least ) but this is what i get
> on the console :
>
> crawler@d1r2n2:/hadoop$ bin/hadoop jar hadoop-test-1.1.1.jar  TestDFSIO
> -write -nrFiles 10 -fileSize 1000
> TestDFSIO.0.0.4
> 13/04/19 17:23:43 INFO fs.TestDFSIO: nrFiles = 10
> 13/04/19 17:23:43 INFO fs.TestDFSIO: fileSize (MB) = 1000
> 13/04/19 17:23:43 INFO fs.TestDFSIO: bufferSize = 1000000
> 13/04/19 17:23:43 INFO fs.TestDFSIO: creating control file: 1000 mega
> bytes, 10 files
> 13/04/19 17:23:44 INFO fs.TestDFSIO: created control files for: 10 files
> 13/04/19 17:23:44 INFO mapred.FileInputFormat: Total input paths to
> process : 10
> 13/04/19 17:23:44 INFO mapred.JobClient: Running job: job_201304191712_0002
> 13/04/19 17:23:45 INFO mapred.JobClient:  map 0% reduce 0%
> 13/04/19 17:24:06 INFO mapred.JobClient:  map 20% reduce 0%
> 13/04/19 17:24:07 INFO mapred.JobClient:  map 30% reduce 0%
> 13/04/19 17:24:09 INFO mapred.JobClient:  map 50% reduce 0%
> 13/04/19 17:24:11 INFO mapred.JobClient:  map 60% reduce 0%
> 13/04/19 17:24:12 INFO mapred.JobClient:  map 90% reduce 0%
> 13/04/19 17:24:13 INFO mapred.JobClient:  map 100% reduce 0%
> 13/04/19 17:24:21 INFO mapred.JobClient:  map 100% reduce 33%
> 13/04/19 17:24:22 INFO mapred.JobClient:  map 100% reduce 100%
> 13/04/19 17:24:23 INFO mapred.JobClient: Job complete:
> job_201304191712_0002
> 13/04/19 17:24:23 INFO mapred.JobClient: Counters: 33
> 13/04/19 17:24:23 INFO mapred.JobClient:   Job Counters
> 13/04/19 17:24:23 INFO mapred.JobClient:     Launched reduce tasks=1
> 13/04/19 17:24:23 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=210932
> 13/04/19 17:24:23 INFO mapred.JobClient:     Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/04/19 17:24:23 INFO mapred.JobClient:     Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/04/19 17:24:23 INFO mapred.JobClient:     Rack-local map tasks=2
> 13/04/19 17:24:23 INFO mapred.JobClient:     Launched map tasks=10
> 13/04/19 17:24:23 INFO mapred.JobClient:     Data-local map tasks=8
> 13/04/19 17:24:23 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=8650
> 13/04/19 17:24:23 INFO mapred.JobClient:   File Input Format Counters
> 13/04/19 17:24:23 INFO mapred.JobClient:     Bytes Read=1120
> 13/04/19 17:24:23 INFO mapred.JobClient:   SkippingTaskCounters
> 13/04/19 17:24:23 INFO mapred.JobClient:     MapProcessedRecords=10
> 13/04/19 17:24:23 INFO mapred.JobClient:     ReduceProcessedGroups=5
> 13/04/19 17:24:23 INFO mapred.JobClient:   File Output Format Counters
> 13/04/19 17:24:23 INFO mapred.JobClient:     Bytes Written=79
> 13/04/19 17:24:23 INFO mapred.JobClient:   FileSystemCounters
> 13/04/19 17:24:23 INFO mapred.JobClient:     FILE_BYTES_READ=871
> 13/04/19 17:24:23 INFO mapred.JobClient:     HDFS_BYTES_READ=2330
> 13/04/19 17:24:23 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=272508
> 13/04/19 17:24:23 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=10485760079
> 13/04/19 17:24:23 INFO mapred.JobClient:   Map-Reduce Framework
> 13/04/19 17:24:23 INFO mapred.JobClient:     Map output materialized
> bytes=925
> 13/04/19 17:24:23 INFO mapred.JobClient:     Map input records=10
> 13/04/19 17:24:23 INFO mapred.JobClient:     Reduce shuffle bytes=925
> 13/04/19 17:24:23 INFO mapred.JobClient:     Spilled Records=100
> 13/04/19 17:24:23 INFO mapred.JobClient:     Map output bytes=765
> 13/04/19 17:24:23 INFO mapred.JobClient:     Total committed heap usage
> (bytes)=7996702720
> 13/04/19 17:24:23 INFO mapred.JobClient:     CPU time spent (ms)=104520
> 13/04/19 17:24:23 INFO mapred.JobClient:     Map input bytes=260
> 13/04/19 17:24:23 INFO mapred.JobClient:     SPLIT_RAW_BYTES=1210
> 13/04/19 17:24:23 INFO mapred.JobClient:     Combine input records=0
> 13/04/19 17:24:23 INFO mapred.JobClient:     Reduce input records=50
http://www.lingcc.com