Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> agent.log  I/O exception (java.net.SocketTimeoutException) caught when processing request: Read timed out


+
Kumar, Suresh 2012-09-17, 21:00
+
邵赛赛 2012-09-18, 01:28
+
Kumar, Suresh 2012-09-18, 17:45
Copy link to this message
-
RE: agent.log I/O exception (java.net.SocketTimeoutException) caught when processing request: Read timed out
Hi Suresh,
I met this exception before.
ERROR 1983406566@qtp-1510130526-10 WidgetBean - java.lang.NullPointerException
        at org.apache.hadoop.chukwa.rest.bean.WidgetBean.<init>(WidgetBean.java:55)
        at org.apache.hadoop.chukwa.datastore.WidgetStore.cacheWidgets(WidgetStore.java:98)
        at org.apache.hadoop.chukwa.datastore.WidgetStore.list(WidgetStore.java:121)
        at org.apache.hadoop.chukwa.rest.bean.WidgetBean.update(WidgetBean.java:158)
        at org.apache.hadoop.chukwa.rest.bean.ColumnBean.update(ColumnBean.java:63)
       at org.apache.hadoop.chukwa.rest.bean.PagesBean.update(PagesBean.java:83)
        at org.apache.hadoop.chukwa.rest.bean.ViewBean.update(ViewBean.java:127)
        at org.apache.hadoop.chukwa.datastore.ViewStore.load(ViewStore.java:92)
        at org.apache.hadoop.chukwa.datastore.ViewStore.<init>(ViewStore.java:61)
        at org.apache.hadoop.chukwa.rest.resource.ViewResource.getView(ViewResource.java:52)
I think this is a bug in HICC while reading descriptors from HDFS, at that time I delete this file (/chukwa/hicc/widgets/disabled) on hdfs and restart HICC, you can try it.
I’m sorry for the other Zookeeper exceptions, also In my environment, HICC only take several minutes to load.

Thanks
Saisai
From: Kumar, Suresh [mailto:[EMAIL PROTECTED]]
Sent: Wednesday, September 19, 2012 1:45 AM
To: [EMAIL PROTECTED]
Subject: RE: agent.log I/O exception (java.net.SocketTimeoutException) caught when processing request: Read timed out

I had checked these before posting, the problem was in the “initial_adaptors” file, I had copied
“add FileTailer FooData /tmp/foo 0” from the Chukwa agent documentation, it didn’t work. I had
to change to  “add filetailer.FileTailingAdaptor FooData /tmp/foo 0” to make it work. I found the
exception in “agent.log” and fixed the problem.

However, localhost:4080/hicc took over 3 hours to load, the hicc.log had this exception, any clues?

Thanks,
Suresh

2012-09-17 18:04:25,698 ERROR 2124414527@qtp-1510130526-7-EventThread ZooKeeperWatcher - ZK is null on connection event -- see stack trace for the stack trace when constructor was called on this zkw
java.lang.Exception: ZKW CONSTRUCTOR STACK TRACE FOR DEBUGGING
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:142)
        at org.apache.hadoop.hbase.zookeeper.ZooKeeperWatcher.<init>(ZooKeeperWatcher.java:126)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.getZooKeeperWatcher(HConnectionManager.java:1307)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.setupZookeeperTrackers(HConnectionManager.java:579)
        at org.apache.hadoop.hbase.client.HConnectionManager$HConnectionImplementation.<init>(HConnectionManager.java:570)
        at org.apache.hadoop.hbase.client.HConnectionManager.getConnection(HConnectionManager.java:187)
        at org.apache.hadoop.hbase.client.HTable.<init>(HTable.java:195)
        at org.apache.hadoop.hbase.client.HTableFactory.createHTableInterface(HTableFactory.java:36)
        at org.apache.hadoop.hbase.client.HTablePool.createHTable(HTablePool.java:268)
        at org.apache.hadoop.hbase.client.HTablePool.findOrCreateTable(HTablePool.java:198)
        at org.apache.hadoop.hbase.client.HTablePool.getTable(HTablePool.java:173)
        at org.apache.hadoop.chukwa.datastore.ChukwaHBaseStore.getClusterNames(ChukwaHBaseStore.java:231)
        at org.apache.hadoop.chukwa.hicc.ClusterConfig.<init>(ClusterConfig.java:63)
        at org.apache.jsp.index_jsp._jspService(org.apache.jsp.index_jsp:99)
        at org.apache.jasper.runtime.HttpJspBase.service(HttpJspBase.java:109)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        at org.apache.jasper.servlet.JspServletWrapper.service(JspServletWrapper.java:389)
        at org.apache.jasper.servlet.JspServlet.serviceJspFile(JspServlet.java:486)
        at org.apache.jasper.servlet.JspServlet.service(JspServlet.java:380)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
        at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:327)
        at org.mortbay.jetty.servlet.Dispatcher.forward(Dispatcher.java:126)
        at org.mortbay.jetty.servlet.DefaultServlet.doGet(DefaultServlet.java:503)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:740)
        at javax.servlet.http.HttpServlet.service(HttpServlet.java:853)
        at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
        at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:401)
        at org.mortbay.jetty.security.SecurityHandler.handle(SecurityHandler.java:216)
        at org.mortbay.jetty.servlet.SessionHandler.handle(SessionHandler.java:182)
        at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:766)
        at org.mortbay.jetty.webapp.WebAppContext.handle(WebAppContext.java:450)
        at org.mortbay.jetty.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:230)
        at org.mortbay.jetty.handler.HandlerCollection.handle(HandlerCollection.java:114)
        at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
        at org.mortbay.jetty.Server.handle(Server.java:326)
        at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:542)
        at org.mortbay.jetty.HttpConnection$RequestHandler.headerComplete(HttpConnect
+
Kumar, Suresh 2012-09-19, 20:26
+
Shao, Saisai 2012-09-20, 00:10
+
Kumar, Suresh 2012-09-21, 22:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB