Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> chukwa for processing log4j logs


Copy link to this message
-
Re: chukwa for processing log4j logs
I was able to get the chukwa metrics fine. But, still something is not right
with Hadoop metrics.

I get this error in agent.log :

[root@ip-10-176-22-21 chukwa-0.5.0]# log4j:ERROR Could not connect to remote
log4j server at [localhost]. We will try again later.
java.net.ConnectException: Connection refused
        at java.net.PlainSocketImpl.socketConnect(Native Method)
        at java.net.PlainSocketImpl.doConnect(PlainSocketImpl.java:333)
        at
java.net.PlainSocketImpl.connectToAddress(PlainSocketImpl.java:195)
        at java.net.PlainSocketImpl.connect(PlainSocketImpl.java:182)
        at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:366)
        at java.net.Socket.connect(Socket.java:529)
        at java.net.Socket.connect(Socket.java:478)
        at java.net.Socket.<init>(Socket.java:375)
        at java.net.Socket.<init>(Socket.java:218)
        at
org.apache.log4j.net.SocketAppender.connect(SocketAppender.java:203)
        at
org.apache.log4j.net.SocketAppender.<init>(SocketAppender.java:153)
        at
org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext.emitRecord(Log4JMetricsContext.java:91)
        at
org.apache.hadoop.metrics.spi.AbstractMetricsContext.emitRecords(AbstractMetricsContext.java:306)
        at
org.apache.hadoop.metrics.spi.AbstractMetricsContext.timerEvent(AbstractMetricsContext.java:292)
        at
org.apache.hadoop.metrics.spi.AbstractMetricsContext.access$000(AbstractMetricsContext.java:52)
        at
org.apache.hadoop.metrics.spi.AbstractMetricsContext$1.run(AbstractMetricsContext.java:251)
        at java.util.TimerThread.mainLoop(Timer.java:512)
        at java.util.TimerThread.run(Timer.java:462)

And, this exception in hadoop's *.out logs for all three - jvm,rpc and dfs:

2011-07-25 20:16:50,319 ERROR [main]
[hadoop.metrics.MetricsUtil.getContext()] - Unable to create metrics context
jvm
java.lang.ClassNotFoundException:
org.apache.hadoop.chukwa.inputtools.log4j.Log4JMetricsContext
        at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
        at java.security.AccessController.doPrivileged(Native Method)
        at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
        at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
        at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
        at java.lang.Class.forName0(Native Method)
        at java.lang.Class.forName(Class.java:169)
        at
org.apache.hadoop.metrics.ContextFactory.getContext(ContextFactory.java:132)
        at
org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:56)
        at
org.apache.hadoop.metrics.MetricsUtil.getContext(MetricsUtil.java:45)
        at
org.apache.hadoop.metrics.jvm.JvmMetrics.<init>(JvmMetrics.java:81)
        at org.apache.hadoop.metrics.jvm.JvmMetrics.init(JvmMetrics.java:73)
        at org.apache.hadoop.metrics.jvm.JvmMetrics.init(JvmMetrics.java:60)
        at
org.apache.hadoop.hdfs.server.datanode.metrics.DataNodeMetrics.<init>(DataNodeMetrics.java:98)
        at
org.apache.hadoop.hdfs.server.datanode.DataNode.instantiateDataNode(DataNode.java:1238)
        at
org.apache.hadoop.hdfs.server.datanode.DataNode.createDataNode(DataNode.java:1246)
        at
org.apache.hadoop.hdfs.server.datanode.DataNode.main(DataNode.java:1368)

Any idea with the ConnectException and which classpath I'm missing?

Thanks,
Abhijit

On Fri, Jul 22, 2011 at 6:14 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> Try restore the original Hadoop log4j.properties, and keep
> hadoop-metrics.properties in HADOOP_CONF_DIR, and jar files in the respected
> places.
> Restart the cluster and check for exception in hadoop log and out files.
>
> regards,
> Eric
>
> On Jul 22, 2011, at 5:36 PM, abhijitdhar wrote:
>
> > I have added add SocketAdaptor Hadoop 9096 0 too. And have the jars added
> to
> > HADOOP_HOME/lib as you mentioned. But, still dont see any logs in agent.
> I'm
> > running hadoop 0.20.4. I'm able to see logs from SystemMetrics adapter
Абижит Дар
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB