Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - java.lang.OutOfMemoryError: PermGen space when running as a service.


Copy link to this message
-
java.lang.OutOfMemoryError: PermGen space when running as a service.
Bennie Schut 2010-06-03, 09:20
Hi guys,

When I run hive as a service like this: "hive --service hiveserver"

I get these errors after about a day of running a lot of queries:
java.lang.OutOfMemoryError: PermGen space

I use:
trunk hive from about a week ago with "-XX:MaxPermSize=128m".
hadoop 0.20.2
mysql 5.1.45 meta store
Some UDF's on each query.

I've now increased it to 512 to see if it helps.

I've also made a jmap dump (85M) while it was broken
9931 instances of class org.apache.hadoop.hive.ql.exec.ColumnInfo
4466 instances of class org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc
3127 instances of class org.apache.hadoop.hive.metastore.api.FieldSchema
3127 instances of class org.apache.hadoop.hive.metastore.api.FieldSchema$Isset
2755 instances of class org.apache.commons.logging.impl.Log4JLogger
2497 instances of class org.datanucleus.util.WeakValueMap$WeakValueReference
1404 instances of class org.apache.hadoop.ipc.Client$Call
1404 instances of class org.apache.hadoop.ipc.RPC$Invocation
1394 instances of class org.apache.hadoop.ipc.RemoteException
985 instances of class [Lorg.datanucleus.plugin.ConfigurationElement;
945 instances of class org.datanucleus.plugin.ConfigurationElement
907 instances of class org.apache.hadoop.hive.ql.hooks.LineageInfo$BaseColumnInfo
792 instances of class [Lorg.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
767 instances of class org.apache.hadoop.hive.ql.hooks.LineageInfo$Dependency
730 instances of class org.antlr.runtime.BitSet
618 instances of class com.mysql.jdbc.ConnectionPropertiesImpl$BooleanConnectionProperty
523 instances of class org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$IdentityConverter
500 instances of class org.datanucleus.sco.backed.Map
464 instances of class org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge

I'm not sure if this is what you expect to see or if something is sticking out here?

Thanks in advance for any ideas on this.
Bennie.