Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> OOMs in java hadoop tests?


Copy link to this message
-
OOMs in java hadoop tests?
Anyone else seeing OutOfMemory errors when trying to run the hadoop tests?

Specifically:
    [junit] Running org.apache.avro.mapred.TestWordCountGeneric
    [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.metrics.jvm.JvmMetrics init
    [junit] INFO: Initializing JVM Metrics with
processName=JobTracker, sessionId    [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: No job jar file set.  User classes may not be
found. See JobConf(Class) or JobConf#setJar(String).
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Running job: job_local_0001
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.MapTask runOldMapper
    [junit] INFO: numReduceTasks: 1
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
    [junit] INFO: io.sort.mb = 100
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.LocalJobRunner$Job run
    [junit] WARNING: job_local_0001
    [junit] java.lang.OutOfMemoryError: Java heap space
    [junit] at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:781)
    [junit] at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
    [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    [junit] at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO:  map 0% reduce 0%
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Job complete: job_local_0001
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.Counters log
    [junit] INFO: Counters: 0
    [junit] Tests run: 1, Failures: 0, Errors: 1, Time elapsed: 2.29 sec
    [junit] Error:
    [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.metrics.jvm.JvmMetrics init
    [junit] INFO: Initializing JVM Metrics with
processName=JobTracker, sessionId    [junit] Apr 3, 2010 5:57:56 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: No job jar file set.  User classes may not be
found. See JobConf(Class) or JobConf#setJar(String).
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Running job: job_local_0001
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:57 PM org.apache.hadoop.mapred.MapTask runOldMapper
    [junit] INFO: numReduceTasks: 1
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
    [junit] INFO: io.sort.mb = 100
    [junit] Apr 3, 2010 5:57:57 PM
org.apache.hadoop.mapred.LocalJobRunner$Job run
    [junit] WARNING: job_local_0001
    [junit] java.lang.OutOfMemoryError: Java heap space
    [junit] at
org.apache.hadoop.mapred.MapTask$MapOutputBuffer.<init>(MapTask.java:781)
    [junit] at org.apache.hadoop.mapred.MapTask.runOldMapper(MapTask.java:350)
    [junit] at org.apache.hadoop.mapred.MapTask.run(MapTask.java:307)
    [junit] at
org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:177)
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO:  map 0% reduce 0%
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Job complete: job_local_0001
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.Counters log
    [junit] INFO: Counters: 0
    [junit]
    [junit] TEST org.apache.avro.mapred.TestWordCountGeneric FAILED
    [junit] Running org.apache.avro.mapred.TestWordCountSpecific
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.metrics.jvm.JvmMetrics init
    [junit] INFO: Cannot initialize JVM Metrics with
processName=JobTracker, sessionId= - already initialized
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: Use GenericOptionsParser for parsing the
arguments. Applications should implement Tool for the same.
    [junit] Apr 3, 2010 5:57:58 PM org.apache.hadoop.mapred.JobClient
configureCommandLineOptions
    [junit] WARNING: No job jar file set.  User classes may not be
found. See JobConf(Class) or JobConf#setJar(String).
    [junit] Apr 3, 2010 5:57:58 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:59 PM org.apache.hadoop.mapred.JobClient
monitorAndPrintJob
    [junit] INFO: Running job: job_local_0002
    [junit] Apr 3, 2010 5:57:59 PM
org.apache.hadoop.mapred.FileInputFormat listStatus
    [junit] INFO: Total input paths to process : 1
    [junit] Apr 3, 2010 5:57:59 PM org.apache.hadoop.mapred.MapTask runOldMapper
    [junit] INFO: numReduceTasks: 1
    [junit] Apr 3, 2010 5:57:59 PM
org.apache.hadoop.mapred.MapTask$MapOutputBuffer <init>
    [junit] INFO: io.sort.mb = 100
    [junit] Apr 3, 2010 5:57:59 PM
org.apache.hadoop.mapred.LocalJobRunner$Job
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB