Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> Problem in collecting data from Agent


+
VenkataHariShankar.S@... 2010-11-19, 06:59
Copy link to this message
-
Re: Problem in collecting data from Agent
You apparently have your agent set to use the console connector. This
means that the agent is writing to standard out, not to HDFS. Did you
specify "local" when starting the agent?

--Ari

On Thu, Nov 18, 2010 at 10:59 PM, <[EMAIL PROTECTED]> wrote:
>
> Hi All,
>
>
>
> When I try to send a log file from Agent to Collector, I didn’t get anything in HDFS expect following two file,(it also have no data inside that)
>
> Please let me know, how to monitor wheather collector is receiving data..
>
>
>
> $ ./hadoop fs -ls /chukwa/logs/
>
> Found 2 items
>
> -rw-r--r--   3 hadoop supergroup          0 2010-11-19 17:14 /chukwa/logs/201019171406118_HADOOPSQOOP_4b2656712c63c0713a7fea.chukwa
>
> -rw-r--r--   3 hadoop supergroup        103 2010-11-19 17:25 /chukwa/logs/201019172022791_HADOOPSQOOP_1d40ead912c63f88f598000.done
>
>
>
> I was trying to send Iostat.log file which is given in chukwa/test/samples directory.
>
>
>
>
>
> Using the adapter
>
>
>
> add filetailer.FileTailingAdaptor logs /tmp/Iostat.log 0
>
> OK add completed; new ID is 4b08ea7f0635fff5d4de778f6782bd1b
>
>
>
> Collector.log Output:
>
> 2010-11-19 17:44:50,418 INFO Timer-3 LocalWriter - stat:datacollection.writer.local.LocalWriter dataSize=0 dataRate=0
>
> 2010-11-19 17:45:18,183 INFO Timer-1 root - stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
>
> 2010-11-19 17:45:19,243 INFO Timer-2 LocalWriter - start Date [Fri Nov 19 17:45:19 IST 2010]
>
> 2010-11-19 17:45:19,243 INFO Timer-2 LocalWriter - Rotate from Timer-2
>
> 2010-11-19 17:45:19,247 INFO Timer-2 LocalWriter - no chunks written to /tmp/chukwa/dataSink/201019174019240_HADOOPSQOOP_4c320ac712c63fd13837ffa.chukwa, deleting
>
> 2010-11-19 17:45:20,419 INFO Timer-3 LocalWriter - stat:datacollection.writer.local.LocalWriter dataSize=0 dataRate=0
>
> 2010-11-19 17:45:50,420 INFO Timer-3 LocalWriter - stat:datacollection.writer.local.LocalWriter dataSize=0 dataRate=0
>
> 2010-11-19 17:46:18,184 INFO Timer-1 root - stats:ServletCollector,numberHTTPConnection:0,numberchunks:0
>
> 2010-11-19 17:46:20,420 INFO Timer-3 LocalWriter - stat:datacollection.writer.local.LocalWriter dataSize=0 dataRate=0
>
>
>
> Agent Console Output:
>
> …….
>
> data type was logs
>
> data length was 131072, not printing
>
> -------------------
>
> Console out connector got event at offset 1441792
>
> data type was logs
>
> data length was 131072, not printing
>
> -------------------
>
> Console out connector got event at offset 1572864
>
> data type was logs
>
> data length was 131072, not printing
>
> -------------------
>
> Console out connector got event at offset 1703936
>
> data type was logs
>
> data length was 131072, not printing
>
> -------------------
>
> Console out connector got event at offset 1740827
>
> data type was logs
>
> data length was 36891, not printing
>
>
>
> Thanks & Regards
>
> Venkat
--
Ari Rabkin [EMAIL PROTECTED]
UC Berkeley Computer Science Department
+
Stuti Awasthi 2010-11-19, 07:59