Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa, mail # user - How to check whether chukwa collecting logs


Copy link to this message
-
How to check whether chukwa collecting logs
Shouvanik Haldar 2013-01-09, 10:54
I am trying to find out a check point , so that I can say that Chukwa is
running and collecting logs
I configured the /chukwa/etc/chukwa/initial_adaptors
with following

add FileTailer
/home/ec2-user/hadoop-1.0.4/logs/hadoop-ec2-user-datanode-domU-12-31-39-07-19-3B.log
0

.
then I started bin/chukwa agent.

Started the hadoop daemons.

then, bin/chukwa collector.

Next, I check the site
"http://ec2-204-236-192-203.compute-1.amazonaws.com:8080/chukwa?ping=true"

with following statistics

Date:1357728723103
Now:1357728765604
numberHTTPConnection in time window:17
numberchunks in time window:119
lifetimechunks:23029
and when I check hdfs I see

-rw-r--r--   1 ec2-user supergroup     657818 2013-01-08 11:09
/chukwa/logs/201308110441175_domU12313907193B_216836a313c19d565ee8000.done
-rw-r--r--   1 ec2-user supergroup     483520 2013-01-08 11:14
/chukwa/logs/201308110941296_domU12313907193B_216836a313c19d565ee7ffe.done
-rw-r--r--   1 ec2-user supergroup     483155 2013-01-08 11:19
/chukwa/logs/201308111441296_domU12313907193B_216836a313c19d565ee7ffc.done
-rw-r--r--   1 ec2-user supergroup     463708 2013-01-08 11:24
/chukwa/logs/201308111941296_domU12313907193B_216836a313c19d565ee7ffa.done
-rw-r--r--   1 ec2-user supergroup     386337 2013-01-08 11:36
/chukwa/logs/201308113134948_domU12313907193B_488afeb413c19ee05bf8000.done
-rw-r--r--   1 ec2-user supergroup     479656 2013-01-08 11:41
/chukwa/logs/201308113635034_domU12313907193B_488afeb413c19ee05bf7ffe.done
-rw-r--r--   1 ec2-user supergroup     487748 2013-01-08 11:46
/chukwa/logs/201308114135034_domU12313907193B_488afeb413c19ee05bf7ffb.done
-rw-r--r--   1 ec2-user supergroup     479696 2013-01-08 11:51
/chukwa/logs/201308114635034_domU12313907193B_488afeb413c19ee05bf7ff8.done
-rw-r--r--   1 ec2-user supergroup     488030 2013-01-08 11:56
/chukwa/logs/201308115135034_domU12313907193B_488afeb413c19ee05bf7ff5.done
-rw-r--r--   1 ec2-user supergroup     479754 2013-01-08 12:01
/chukwa/logs/201308115635034_domU12313907193B_488afeb413c19ee05bf7ff2.done
-rw-r--r--   1 ec2-user supergroup     488067 2013-01-08 12:06
/chukwa/logs/201308120135034_domU12313907193B_488afeb413c19ee05bf7fef.done
-rw-r--r--   1 ec2-user supergroup     479961 2013-01-08 12:11
/chukwa/logs/201308120635034_domU12313907193B_488afeb413c19ee05bf7fec.done
-rw-r--r--   1 ec2-user supergroup     487976 2013-01-08 12:16
/chukwa/logs/201308121135034_domU12313907193B_488afeb413c19ee05bf7fe9.done
-rw-r--r--   1 ec2-user supergroup     479863 2013-01-08 12:21
/chukwa/logs/201308121635034_domU12313907193B_488afeb413c19ee05bf7fe6.done
-rw-r--r--   1 ec2-user supergroup     649168 2013-01-08 12:42
/chukwa/logs/201308123755390_domU12313907193B_5ab1511313c1a2ac24f8000.done
-rw-r--r--   1 ec2-user supergroup     483972 2013-01-08 12:47
/chukwa/logs/201308124255466_domU12313907193B_5ab1511313c1a2ac24f7ffc.done
-rw-r--r--   1 ec2-user supergroup     483417 2013-01-08 12:52
/chukwa/logs/201308124755466_domU12313907193B_5ab1511313c1a2ac24f7ff8.done
-rw-r--r--   1 ec2-user supergroup     576711 2013-01-08 13:06
/chukwa/logs/201308130134839_domU12313907193B_2abeddbf13c1a406b0b8000.done
-rw-r--r--   1 ec2-user supergroup     483877 2013-01-08 13:11
/chukwa/logs/201308130634912_domU12313907193B_2abeddbf13c1a406b0b7ffc.done
-rw-r--r--   1 ec2-user supergroup     484352 2013-01-08 13:16
/chukwa/logs/201308131134912_domU12313907193B_2abeddbf13c1a406b0b7ff7.done
-rw-r--r--   1 ec2-user supergroup     486158 2013-01-08 13:21
/chukwa/logs/201308131634912_domU12313907193B_2abeddbf13c1a406b0b7ff2.done
-rw-r--r--   1 ec2-user supergroup     483925 2013-01-08 13:26
/chukwa/logs/201308132134913_domU12313907193B_2abeddbf13c1a406b0b7fed.done
-rw-r--r--   1 ec2-user supergroup     629788 2013-01-08 13:37
/chukwa/logs/201308133253308_domU12313907193B_7382697b13c1a5d14db8000.done
-rw-r--r--   1 ec2-user supergroup     485716 2013-01-08 13:42
/chukwa/logs/201308133753394_domU12313907193B_7382697b13c1a5d14db7ffb.done
-rw-r--r--   1 ec2-user supergroup     486475 2013-01-08 13:47
/chukwa/logs/201308134253394_domU12313907193B_7382697b13c1a5d14db7ff5.done
-rw-r--r--   1 ec2-user supergroup     481330 2013-01-08 13:52
/chukwa/logs/201308134753394_domU12313907193B_7382697b13c1a5d14db7fef.done
-rw-r--r--   1 ec2-user supergroup     488138 2013-01-08 13:57
/chukwa/logs/201308135253394_domU12313907193B_7382697b13c1a5d14db7fe9.done
-rw-r--r--   1 ec2-user supergroup     484294 2013-01-08 14:02
/chukwa/logs/201308135753394_domU12313907193B_7382697b13c1a5d14db7fe3.done
-rw-r--r--   1 ec2-user supergroup     480689 2013-01-08 14:07
/chukwa/logs/201308140253394_domU12313907193B_7382697b13c1a5d14db7fdd.done
-rw-r--r--   1 ec2-user supergroup     488018 2013-01-08 14:12
/chukwa/logs/201308140753395_domU12313907193B_7382697b13c1a5d14db7fd7.done
-rw-r--r--   1 ec2-user supergroup     287583 2013-01-08 14:23
/chukwa/logs/201308141253395_domU12313907193B_7382697b13c1a5d14db7fd1.done
-rw-r--r--   1 ec2-user supergroup     527781 2013-01-08 14:27
/chukwa/logs/201308142254785_domU12313907193B_3065fa8513c1a8ae1598000.done
-rw-r--r--   1 ec2-user supergroup     483844 2013-01-08 14:32
/chukwa/logs/201308142754897_domU12313907193B_3065fa8513c1a8ae1597ff9.done
-rw-r--r--   1 ec2-user supergroup     483447 2013-01-08 14:37
/chukwa/logs/201308143254897_domU12313907193B_3065fa8513c1a8ae1597ff3.done
-rw-r--r--   1 ec2-user supergroup     483915 2013-01-08 14:42
/chukwa/logs/201308143754897_domU12313907193B_3065fa8513c1a8ae1597fed.done
-rw-r--r--   1 ec2-user supergroup     483902 2013-01-08 14:47
/chukwa/logs/201308144254897_domU12313907193B_3065fa8513c1a8ae1597fe7.done
-rw-r--r--   1 ec2-user supergroup     484187 2013-01-08 14:52
/chukwa/logs/201308144754897_domU12313907193B_3065fa8513c1a8ae1597fe1.done
-rw-r--r--   1 ec2-user supergroup     295291 2013-01-08 17:04
/chukwa/logs/201308145254897_domU12313907193B_3065fa8513c1a8ae1597fdb.done
-rw-r--r--   1 ec2-user supergroup     524107 2013-01-08 17:08
/chukwa/logs/201308170310783_domU12313
+
Shouvanik Haldar 2013-01-10, 07:11
+
Eric Yang 2013-01-10, 06:25
+
Shouvanik Haldar 2013-01-10, 07:01
+
shreyas subramanya 2013-01-10, 07:24
+
Shouvanik Haldar 2013-01-10, 07:34
+
shreyas subramanya 2013-01-10, 07:44
+
Shouvanik Haldar 2013-01-10, 08:06
+
Shouvanik Haldar 2013-01-10, 09:42