Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # dev >> Need help to configure chukwa to write logs file generated by log4j to hdfs


+
NehaS Singh 2012-12-26, 06:19
+
Eric Yang 2012-12-26, 18:06
Copy link to this message
-
Re: Need help to configure chukwa to write logs file generated by log4j to hdfs
I have added the HBASE_CONF_DIR as well as HADOOP_CONF_DIR to the classpath
and my collector.og file looks as below
2012-12-20 16:43:43,919 INFO main ChukwaConfiguration - chukwaConf is
/home/user/chukwa-0.5.0/chukwa-incubating-0.5.0/etc/chukwa
2012-12-20 16:43:44,244 INFO main log - Logging to
org.slf4j.impl.Log4jLoggerAdapter(org.mortbay.log) via
org.mortbay.log.Slf4jLog
2012-12-20 16:43:44,288 INFO main log - jetty-6.1.26
2012-12-20 16:43:44,331 INFO main root - initing servletCollector
2012-12-20 16:43:44,334 INFO main SeqFileWriter - rotateInterval is 300000
2012-12-20 16:43:44,334 INFO main SeqFileWriter - not using fixed time
interval scheme
2012-12-20 16:43:44,334 INFO main SeqFileWriter - outputDir is
/home/user/chukwa-0.5.0/chukwa-incubating-0.5.0/chukwa/logs
2012-12-20 16:43:44,334 INFO main SeqFileWriter - fsname is
hdfs://172.20.104.226:8020
2012-12-20 16:43:44,334 INFO main SeqFileWriter - filesystem type from
core-default.xml is org.apache.hadoop.hdfs.DistributedFileSystem
2012-12-20 16:43:44,816 INFO main log - Started
SelectChannelConnector@0.0.0.0:9080
2012-12-20 16:43:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:43:45,805 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:44:15,805 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:44:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:44:45,805 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:45:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:45:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:45:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:46:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:46:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:46:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:47:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:47:45,331 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:47:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:48:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:48:44,805 INFO Timer-2 SeqFileWriter - closing sink
file/home/user/chukwa-0.5.0/chukwa-incubating-0.5.0/chukwa/logs/20121220164344752_INFPH01464U_1de46d0$
2012-12-20 16:48:44,855 INFO Timer-2 SeqFileWriter - no chunks written to
/home/user/chukwa-0.5.0/chukwa-incubating-0.5.0/chukwa/logs/20121220164344752_INFPH01464U_1de$
2012-12-20 16:48:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:48:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:49:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:49:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:49:45,807 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:50:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:50:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:50:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:51:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:51:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
2012-12-20 16:51:45,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:52:15,806 INFO Timer-3 SeqFileWriter -
stat:datacollection.writer.hdfs dataSize=0 dataRate=0
2012-12-20 16:52:45,332 INFO Timer-1 root -
stats:ServletCollector,numberHTTPConnection:0,numberchunks:0

I am not able to understand the error.could u please help.

View this message in context: http://apache-chukwa.679492.n3.nabble.com/Need-help-to-configure-chukwa-to-write-logs-file-generated-by-log4j-to-hdfs-tp4025511p4025515.html
Sent from the Chukwa - Developers mailing list archive at Nabble.com.
+
Eric Yang 2012-12-27, 09:06