Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # dev - OOMs in java hadoop tests?


Copy link to this message
-
OOMs in java hadoop tests?
Jeff Hodges 2010-04-04, 01:09
Anyone else seeing OutOfMemory errors when trying to run the hadoop tests?

Specifically:
    [junit] Running org.apache.avro.mapred.TestWordCountGeneric
    [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.metrics.jvm.JvmMetrics init
    [junit] INFO: Initializing JVM Metrics with
processName=JobTracker, sessionId    [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: No job jar file set.  User classes may not be
found. See JobConf(Class) or JobConf#setJar(String).
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Running job: job_local_0001
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.MapTask runOldMapper
    [junit] INFO: numReduceTasks: 1
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
    [junit] INFO: io.sort.mb = 100
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.LocalJobRunner$Job run
    [junit] WARNING: job_local_0001
    [junit] java.lang.OutOfMemoryError: Java heap space
    [junit] at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:781)
    [junit] at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
    [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    [junit] at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO:  map 0% reduce 0%
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Job complete: job_local_0001
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.Counters log
    [junit] INFO: Counters: 0
    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 2.29 sec
    [junit] Error:
    [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.metrics.jvm.JvmMetrics init
    [junit] INFO: Initializing JVM Metrics with
processName=JobTracker, sessionId    [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: No job jar file set.  User classes may not be
found. See JobConf(Class) or JobConf#setJar(String).
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Running job: job_local_0001
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.MapTask runOldMapper
    [junit] INFO: numReduceTasks: 1
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
    [junit] INFO: io.sort.mb = 100
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.LocalJobRunner$Job run
    [junit] WARNING: job_local_0001
    [junit] java.lang.OutOfMemoryError: Java heap space
    [junit] at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:781)
    [junit] at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
    [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    [junit] at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO:  map 0% reduce 0%
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Job complete: job_local_0001
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.Counters log
    [junit] INFO: Counters: 0
    [junit]
    [junit] TEST org.apache.avro.mapred.TestWordCountGeneric FAILED
    [junit] Running org.apache.avro.mapred.TestWordCountSpecific
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.metrics.jvm.JvmMetrics init
    [junit] INFO: Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: No job jar file set.  User classes may not be
found. See JobConf(Class) or JobConf#setJar(String).
    [junit] Apr 3, 2010 5:57:58 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:59 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Running job: job_local_0002
    [junit] Apr 3, 2010 5:57:59 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:59 PM org.apache.hadoop.mapred.MapTask runOldMapper
    [junit] INFO: numReduceTasks: 1
    [junit] Apr 3, 2010 5:57:59 PM
org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
    [junit] INFO: io.sort.mb = 100
    [junit] Apr 3, 2010 5:57:59 PM
org.apache.hadoop.mapred.LocalJobRunner$Job