Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> java.lang.OutOfMemoryError: PermGen space when running as a service.


Copy link to this message
-
java.lang.OutOfMemoryError: PermGen space when running as a service.
Hi guys,

When I run hive as a service like this: "hive --service hiveserver"

I get these errors after about a day of running a lot of queries:
java.lang.OutOfMemoryError: PermGen space

I use:
trunk hive from about a week ago with "-XX:MaxPermSize=128m".
hadoop 0.20.2
mysql 5.1.45 meta store
Some UDF's on each query.

I've now increased it to 512 to see if it helps.

I've also made a jmap dump (85M) while it was broken
9931 instances of class org.apache.hadoop.hive.ql.exec.ColumnInfo
4466 instances of class org.apache.hadoop.hive.ql.plan.ExprNodeColumnDesc
3127 instances of class org.apache.hadoop.hive.metastore.api.FieldSchema
3127 instances of class org.apache.hadoop.hive.metastore.api.FieldSchema$Isset
2755 instances of class org.apache.commons.logging.impl.Log4JLogger
2497 instances of class org.datanucleus.util.WeakValueMap$WeakValueReference
1404 instances of class org.apache.hadoop.ipc.Client$Call
1404 instances of class org.apache.hadoop.ipc.RPC$Invocation
1394 instances of class org.apache.hadoop.ipc.RemoteException
985 instances of class [Lorg.datanucleus.plugin.ConfigurationElement;
945 instances of class org.datanucleus.plugin.ConfigurationElement
907 instances of class org.apache.hadoop.hive.ql.hooks.LineageInfo$BaseColumnInfo
792 instances of class [Lorg.apache.hadoop.hive.serde2.objectinspector.ObjectInspector;
767 instances of class org.apache.hadoop.hive.ql.hooks.LineageInfo$Dependency
730 instances of class org.antlr.runtime.BitSet
618 instances of class com.mysql.jdbc.ConnectionPropertiesImpl$BooleanConnectionProperty
523 instances of class org.apache.hadoop.hive.serde2.objectinspector.ObjectInspectorConverters$IdentityConverter
500 instances of class org.datanucleus.sco.backed.Map
464 instances of class org.apache.hadoop.hive.ql.udf.generic.GenericUDFBridge

I'm not sure if this is what you expect to see or if something is sticking out here?

Thanks in advance for any ideas on this.
Bennie.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB