Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # dev >> Need help to configure chukwa to write logs file generated by log4j to hdfs


Copy link to this message
-
Re: Need help to configure chukwa to write logs file generated by log4j to hdfs
I have added the HBASE_CONF_DIR as well as HADOOP_CONF_DIR to the classpath
and my collector.og file looks as below
2012-12-20 16:43:43,919 INFO main ChukwaConfiguration - chukwaConf is
/home/user/chukwa-0.5.0/chukwa-incubating-0.5.0/etc/chukwa
2012-12-20 16:43:44,244 INFO main log - Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
2012-12-20 16:43:44,288 INFO main log - jetty-6.1.26
2012-12-20 16:43:44,331 INFO main root - initing servletCollector
2012-12-20 16:43:44,334 INFO main SeqFileWriter - rotateInterval is 300000
2012-12-20 16:43:44,334 INFO main SeqFileWriter - not using fixed time
interval scheme
2012-12-20 16:43:44,334 INFO main SeqFileWriter - outputDir is
/home/user/chukwa-0.5.0/chukwa-incubating-0.5.0/chukwa/logs
2012-12-20 16:43:44,334 INFO main SeqFileWriter - fsname is
hdfs://172.20.104.226:8020
2012-12-20 16:43:44,334 INFO main SeqFileWriter - filesystem type from
core-default.xml is org.apache.hadoop.hdfs.DistributedFileSystem
2012-12-20 16:43:44,816 INFO main log - Started
SelectChannelConnector@0.0.0.0:9080
2012-12-20 16:43:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:43:45,805 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:44:15,805 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:44:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:44:45,805 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:45:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:45:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:45:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:46:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:46:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:46:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:47:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:47:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:47:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:48:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:48:44,805 INFO Timer-2 SeqFileWriter - closing sink
file/home/user/chukwa-0.5.0/chukwa-incubating-0.5.0/chukwa/logs/20121220164344752_INFPH01464U_1de46d0$
2012-12-20 16:48:44,855 INFO Timer-2 SeqFileWriter - no chunks written to
/home/user/chukwa-0.5.0/chukwa-incubating-0.5.0/chukwa/logs/20121220164344752_INFPH01464U_1de$
2012-12-20 16:48:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:48:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:49:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:49:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:49:45,807 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:50:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:50:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:50:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:51:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:51:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:51:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:52:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:52:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0

I am not able to understand the error.could u please help.

View this message in context: http://apache-chukwa.679492.n3.nabble.com/Need-help-to-configure-chukwa-to-write-logs-file-generated-by-log4j-to-hdfs-tp4025511p4025515.html
Sent from the Chukwa - Developers mailing list archive at Nabble.com.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB