Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> How to check whether chukwa collecting logs


Copy link to this message
-
How to check whether chukwa collecting logs
I am trying to find out a check point , so that I can say that Chukwa is
running and collecting logs
I configured the /chukwa/etc/chukwa/initial_adaptors
with following

add FileTailer
/home/ec2-user/hadoop-1.0.4/logs/hadoop-ec2-user-datanode-domU-12-31-39-07-19-3B.log
0

.
then I started bin/chukwa agent.

Started the hadoop daemons.

then, bin/chukwa collector.

Next, I check the site
"http://ec2-204-236-192-203.compute-1.amazonaws.com:8080/chukwa?ping=true"

with following statistics

Date:1357728723103
Now:1357728765604
numberHTTPConnection in time window:17
numberchunks in time window:119
lifetimechunks:23029
and when I check hdfs I see

-rw-r--r--   1 ec2-user supergroup     657818 2013-01-08 11:09
/chukwa/logs/201308110441175_domU12313907193B_216836a313c19d565ee8000.done
-rw-r--r--   1 ec2-user supergroup     483520 2013-01-08 11:14
/chukwa/logs/201308110941296_domU12313907193B_216836a313c19d565ee7ffe.done
-rw-r--r--   1 ec2-user supergroup     483155 2013-01-08 11:19
/chukwa/logs/201308111441296_domU12313907193B_216836a313c19d565ee7ffc.done
-rw-r--r--   1 ec2-user supergroup     463708 2013-01-08 11:24
/chukwa/logs/201308111941296_domU12313907193B_216836a313c19d565ee7ffa.done
-rw-r--r--   1 ec2-user supergroup     386337 2013-01-08 11:36
/chukwa/logs/201308113134948_domU12313907193B_488afeb413c19ee05bf8000.done
-rw-r--r--   1 ec2-user supergroup     479656 2013-01-08 11:41
/chukwa/logs/201308113635034_domU12313907193B_488afeb413c19ee05bf7ffe.done
-rw-r--r--   1 ec2-user supergroup     487748 2013-01-08 11:46
/chukwa/logs/201308114135034_domU12313907193B_488afeb413c19ee05bf7ffb.done
-rw-r--r--   1 ec2-user supergroup     479696 2013-01-08 11:51
/chukwa/logs/201308114635034_domU12313907193B_488afeb413c19ee05bf7ff8.done
-rw-r--r--   1 ec2-user supergroup     488030 2013-01-08 11:56
/chukwa/logs/201308115135034_domU12313907193B_488afeb413c19ee05bf7ff5.done
-rw-r--r--   1 ec2-user supergroup     479754 2013-01-08 12:01
/chukwa/logs/201308115635034_domU12313907193B_488afeb413c19ee05bf7ff2.done
-rw-r--r--   1 ec2-user supergroup     488067 2013-01-08 12:06
/chukwa/logs/201308120135034_domU12313907193B_488afeb413c19ee05bf7fef.done
-rw-r--r--   1 ec2-user supergroup     479961 2013-01-08 12:11
/chukwa/logs/201308120635034_domU12313907193B_488afeb413c19ee05bf7fec.done
-rw-r--r--   1 ec2-user supergroup     487976 2013-01-08 12:16
/chukwa/logs/201308121135034_domU12313907193B_488afeb413c19ee05bf7fe9.done
-rw-r--r--   1 ec2-user supergroup     479863 2013-01-08 12:21
/chukwa/logs/201308121635034_domU12313907193B_488afeb413c19ee05bf7fe6.done
-rw-r--r--   1 ec2-user supergroup     649168 2013-01-08 12:42
/chukwa/logs/201308123755390_domU12313907193B_5ab1511313c1a2ac24f8000.done
-rw-r--r--   1 ec2-user supergroup     483972 2013-01-08 12:47
/chukwa/logs/201308124255466_domU12313907193B_5ab1511313c1a2ac24f7ffc.done
-rw-r--r--   1 ec2-user supergroup     483417 2013-01-08 12:52
/chukwa/logs/201308124755466_domU12313907193B_5ab1511313c1a2ac24f7ff8.done
-rw-r--r--   1 ec2-user supergroup     576711 2013-01-08 13:06
/chukwa/logs/201308130134839_domU12313907193B_2abeddbf13c1a406b0b8000.done
-rw-r--r--   1 ec2-user supergroup     483877 2013-01-08 13:11
/chukwa/logs/201308130634912_domU12313907193B_2abeddbf13c1a406b0b7ffc.done
-rw-r--r--   1 ec2-user supergroup     484352 2013-01-08 13:16
/chukwa/logs/201308131134912_domU12313907193B_2abeddbf13c1a406b0b7ff7.done
-rw-r--r--   1 ec2-user supergroup     486158 2013-01-08 13:21
/chukwa/logs/201308131634912_domU12313907193B_2abeddbf13c1a406b0b7ff2.done
-rw-r--r--   1 ec2-user supergroup     483925 2013-01-08 13:26
/chukwa/logs/201308132134913_domU12313907193B_2abeddbf13c1a406b0b7fed.done
-rw-r--r--   1 ec2-user supergroup     629788 2013-01-08 13:37
/chukwa/logs/201308133253308_domU12313907193B_7382697b13c1a5d14db8000.done
-rw-r--r--   1 ec2-user supergroup     485716 2013-01-08 13:42
/chukwa/logs/201308133753394_domU12313907193B_7382697b13c1a5d14db7ffb.done
-rw-r--r--   1 ec2-user supergroup     486475 2013-01-08 13:47
/chukwa/logs/201308134253394_domU12313907193B_7382697b13c1a5d14db7ff5.done
-rw-r--r--   1 ec2-user supergroup     481330 2013-01-08 13:52
/chukwa/logs/201308134753394_domU12313907193B_7382697b13c1a5d14db7fef.done
-rw-r--r--   1 ec2-user supergroup     488138 2013-01-08 13:57
/chukwa/logs/201308135253394_domU12313907193B_7382697b13c1a5d14db7fe9.done
-rw-r--r--   1 ec2-user supergroup     484294 2013-01-08 14:02
/chukwa/logs/201308135753394_domU12313907193B_7382697b13c1a5d14db7fe3.done
-rw-r--r--   1 ec2-user supergroup     480689 2013-01-08 14:07
/chukwa/logs/201308140253394_domU12313907193B_7382697b13c1a5d14db7fdd.done
-rw-r--r--   1 ec2-user supergroup     488018 2013-01-08 14:12
/chukwa/logs/201308140753395_domU12313907193B_7382697b13c1a5d14db7fd7.done
-rw-r--r--   1 ec2-user supergroup     287583 2013-01-08 14:23
/chukwa/logs/201308141253395_domU12313907193B_7382697b13c1a5d14db7fd1.done
-rw-r--r--   1 ec2-user supergroup     527781 2013-01-08 14:27
/chukwa/logs/201308142254785_domU12313907193B_3065fa8513c1a8ae1598000.done
-rw-r--r--   1 ec2-user supergroup     483844 2013-01-08 14:32
/chukwa/logs/201308142754897_domU12313907193B_3065fa8513c1a8ae1597ff9.done
-rw-r--r--   1 ec2-user supergroup     483447 2013-01-08 14:37
/chukwa/logs/201308143254897_domU12313907193B_3065fa8513c1a8ae1597ff3.done
-rw-r--r--   1 ec2-user supergroup     483915 2013-01-08 14:42
/chukwa/logs/201308143754897_domU12313907193B_3065fa8513c1a8ae1597fed.done
-rw-r--r--   1 ec2-user supergroup     483902 2013-01-08 14:47
/chukwa/logs/201308144254897_domU12313907193B_3065fa8513c1a8ae1597fe7.done
-rw-r--r--   1 ec2-user supergroup     484187 2013-01-08 14:52
/chukwa/logs/201308144754897_domU12313907193B_3065fa8513c1a8ae1597fe1.done
-rw-r--r--   1 ec2-user supergroup     295291 2013-01-08 17:04
/chukwa/logs/201308145254897_domU12313907193B_3065fa8513c1a8ae1597fdb.done
-rw-r--r--   1 ec2-user supergroup     524107 2013-01-08 17:08
/chukwa/logs/201308170310783_domU12313
+
Shouvanik Haldar 2013-01-10, 07:11
+
Eric Yang 2013-01-10, 06:25
+
Shouvanik Haldar 2013-01-10, 07:01
+
shreyas subramanya 2013-01-10, 07:24
+
Shouvanik Haldar 2013-01-10, 07:34
+
shreyas subramanya 2013-01-10, 07:44
+
Shouvanik Haldar 2013-01-10, 08:06
+
Shouvanik Haldar 2013-01-10, 09:42
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB