Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Exception in Jobtracker (java.lang.OutOfMemoryError: Java heap space)


Copy link to this message
-
Exception in Jobtracker (java.lang.OutOfMemoryError: Java heap space)
Hi,

I'm using Hadoop v1.2.1 and it is running fine so long(3 months) without
any issues.

Suddenly I got the below error in Jobtracker and jobs are failed to run.

Is this issue in JT or TT or Jetty issue?

2014-04-12 02:13:57,618 ERROR org.mortbay.log: EXCEPTION
java.lang.OutOfMemoryError: Java heap space
        at java.util.HashMap.newKeyIterator(HashMap.java:840)
        at java.util.HashMap$KeySet.iterator(HashMap.java:874)
        at java.util.HashSet.iterator(HashSet.java:153)
        at
sun.nio.ch.SelectorImpl.processDeregisterQueue(SelectorImpl.java:127)
        at sun.nio.ch.EPollSelectorImpl.doSelect(EPollSelectorImpl.java:69)
        at sun.nio.ch.SelectorImpl.lockAndDoSelect(SelectorImpl.java:69)
        at sun.nio.ch.SelectorImpl.select(SelectorImpl.java:80)
        at
org.mortbay.io.nio.SelectorManager$SelectSet.doSelect(SelectorManager.java:498)
        at
org.mortbay.io.nio.SelectorManager.doSelect(SelectorManager.java:192)
        at
org.mortbay.jetty.nio.SelectChannelConnector.accept(SelectChannelConnector.java:124)
        at
org.mortbay.jetty.AbstractConnector$Acceptor.run(AbstractConnector.java:708)
        at
org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
2014-04-12 02:26:02,093 ERROR
org.apache.hadoop.security.UserGroupInformation: PriviledgedActionException
as:root cause:java.io.IOException: java.lang.OutOfMemoryError: Java heap
space
2014-04-12 02:25:47,963 ERROR org.mortbay.log: Error for /jobtracker.jsp
java.lang.OutOfMemoryError: Java heap space
        at java.lang.String.toCharArray(String.java:2725)
        at sun.net.www.ParseUtil.encodePath(ParseUtil.java:86)
        at
sun.misc.URLClassPath$JarLoader.checkResource(URLClassPath.java:674)
        at
sun.misc.URLClassPath$JarLoader.getResource(URLClassPath.java:759)
        at sun.misc.URLClassPath.getResource(URLClassPath.java:169)
        at java.net.URLClassLoader$1.run(URLClassLoader.java:194)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
        at
org.apache.hadoop.mapred.jobtracker_jsp._jspService(jobtracker_jsp.java:508)
        at
org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:97)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
        at
org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1221)
        at
org.apache.hadoop.http.HttpServer$QuotingInputFilter.doFilter(HttpServer.java:914)
        at
org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1212)
        at
org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:399)
        at
org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at
org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
 at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at
org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
        at
org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at
org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at
org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at
org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnection.java:928)
        at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:549)
        at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:212)
        at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:404)
        at
org.mortbay.io.nio.SelectChannelEndPoint.run(SelectChannelEndPoint.java:410)
2014-04-12 02:25:47,963 ERROR org.apache.hadoop.mapred.JobHistory: Unable
to move history file to DONE canonical subfolder.
java.lang.OutOfMemoryError: Java heap space

Please help as I'm facing in production.

Regards,
Viswa.J

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB