Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Re: error while running TestDFSIO


Copy link to this message
-
Re: error while running TestDFSIO
I have got the same problem. And also need help.

Ling Kun
On Sat, Apr 20, 2013 at 8:35 AM, kaveh minooie <[EMAIL PROTECTED]> wrote:

> Hi everyone
>
> I am getting this error when i run TestDFSIO. the job actually finishes
> successfully. ( according to jobtracker at least ) but this is what i get
> on the console :
>
> crawler@d1r2n2:/hadoop$ bin/hadoop jar hadoop-test-1.1.1.jar  TestDFSIO
> -write -nrFiles 10 -fileSize 1000
> TestDFSIO.0.0.4
> 13/04/19 17:23:43 INFO fs.TestDFSIO: nrFiles = 10
> 13/04/19 17:23:43 INFO fs.TestDFSIO: fileSize (MB) = 1000
> 13/04/19 17:23:43 INFO fs.TestDFSIO: bufferSize = 1000000
> 13/04/19 17:23:43 INFO fs.TestDFSIO: creating control file: 1000 mega
> bytes, 10 files
> 13/04/19 17:23:44 INFO fs.TestDFSIO: created control files for: 10 files
> 13/04/19 17:23:44 INFO mapred.FileInputFormat: Total input paths to
> process : 10
> 13/04/19 17:23:44 INFO mapred.JobClient: Running job: job_201304191712_0002
> 13/04/19 17:23:45 INFO mapred.JobClient:  map 0% reduce 0%
> 13/04/19 17:24:06 INFO mapred.JobClient:  map 20% reduce 0%
> 13/04/19 17:24:07 INFO mapred.JobClient:  map 30% reduce 0%
> 13/04/19 17:24:09 INFO mapred.JobClient:  map 50% reduce 0%
> 13/04/19 17:24:11 INFO mapred.JobClient:  map 60% reduce 0%
> 13/04/19 17:24:12 INFO mapred.JobClient:  map 90% reduce 0%
> 13/04/19 17:24:13 INFO mapred.JobClient:  map 100% reduce 0%
> 13/04/19 17:24:21 INFO mapred.JobClient:  map 100% reduce 33%
> 13/04/19 17:24:22 INFO mapred.JobClient:  map 100% reduce 100%
> 13/04/19 17:24:23 INFO mapred.JobClient: Job complete:
> job_201304191712_0002
> 13/04/19 17:24:23 INFO mapred.JobClient: Counters: 33
> 13/04/19 17:24:23 INFO mapred.JobClient:   Job Counters
> 13/04/19 17:24:23 INFO mapred.JobClient:     Launched reduce tasks=1
> 13/04/19 17:24:23 INFO mapred.JobClient:     SLOTS_MILLIS_MAPS=210932
> 13/04/19 17:24:23 INFO mapred.JobClient:     Total time spent by all
> reduces waiting after reserving slots (ms)=0
> 13/04/19 17:24:23 INFO mapred.JobClient:     Total time spent by all maps
> waiting after reserving slots (ms)=0
> 13/04/19 17:24:23 INFO mapred.JobClient:     Rack-local map tasks=2
> 13/04/19 17:24:23 INFO mapred.JobClient:     Launched map tasks=10
> 13/04/19 17:24:23 INFO mapred.JobClient:     Data-local map tasks=8
> 13/04/19 17:24:23 INFO mapred.JobClient:     SLOTS_MILLIS_REDUCES=8650
> 13/04/19 17:24:23 INFO mapred.JobClient:   File Input Format Counters
> 13/04/19 17:24:23 INFO mapred.JobClient:     Bytes Read=1120
> 13/04/19 17:24:23 INFO mapred.JobClient:   SkippingTaskCounters
> 13/04/19 17:24:23 INFO mapred.JobClient:     MapProcessedRecords=10
> 13/04/19 17:24:23 INFO mapred.JobClient:     ReduceProcessedGroups=5
> 13/04/19 17:24:23 INFO mapred.JobClient:   File Output Format Counters
> 13/04/19 17:24:23 INFO mapred.JobClient:     Bytes Written=79
> 13/04/19 17:24:23 INFO mapred.JobClient:   FileSystemCounters
> 13/04/19 17:24:23 INFO mapred.JobClient:     FILE_BYTES_READ=871
> 13/04/19 17:24:23 INFO mapred.JobClient:     HDFS_BYTES_READ=2330
> 13/04/19 17:24:23 INFO mapred.JobClient:     FILE_BYTES_WRITTEN=272508
> 13/04/19 17:24:23 INFO mapred.JobClient:     HDFS_BYTES_WRITTEN=10485760079
> 13/04/19 17:24:23 INFO mapred.JobClient:   Map-Reduce Framework
> 13/04/19 17:24:23 INFO mapred.JobClient:     Map output materialized
> bytes=925
> 13/04/19 17:24:23 INFO mapred.JobClient:     Map input records=10
> 13/04/19 17:24:23 INFO mapred.JobClient:     Reduce shuffle bytes=925
> 13/04/19 17:24:23 INFO mapred.JobClient:     Spilled Records=100
> 13/04/19 17:24:23 INFO mapred.JobClient:     Map output bytes=765
> 13/04/19 17:24:23 INFO mapred.JobClient:     Total committed heap usage
> (bytes)=7996702720
> 13/04/19 17:24:23 INFO mapred.JobClient:     CPU time spent (ms)=104520
> 13/04/19 17:24:23 INFO mapred.JobClient:     Map input bytes=260
> 13/04/19 17:24:23 INFO mapred.JobClient:     SPLIT_RAW_BYTES=1210
> 13/04/19 17:24:23 INFO mapred.JobClient:     Combine input records=0
> 13/04/19 17:24:23 INFO mapred.JobClient:     Reduce input records=50
http://www.lingcc.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB