Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> how to print the channel capacity

liuyongbo 2013-05-15, 05:37
Nitin Pawar 2013-05-15, 08:20
Nitin Pawar 2013-05-15, 08:46
Nitin Pawar 2013-05-15, 08:49
liuyongbo 2013-05-15, 11:39
Nitin Pawar 2013-05-15, 11:48
Copy link to this message
RE: how to print the channel capacity
There are a few ways to monitor flume in operation. We use the JSON reporting, which is available via 'http://<agent address>:<port>/metrics'. You need to start the agent with the following parameters to get this interface:

-Dflume.monitoring.type=http -Dflume.monitoring.port=34545

We use cacti to graph channel size both as a percentage of maximum and absolute number of events in channel. This provides warning if the sinks cannot keep up with the sources.

We also graph ingress/egress event counts, much like a network bandwidth graph, for some channels to get an idea of the throughput and to see if sources/sinks are running at same speed.
From: liuyongbo [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, May 14, 2013 10:38 PM
Subject: how to print the channel capacity

         I'm using flume to pass log data to mongodb, but I find that some data lose when the pressure is in high level, so I want to know the max request that flume can hold and need to print the capacity.but I can not find the proper way to do this instead of change the source code. Any ideas?
Matt Wise 2013-05-15, 17:10
Matt Wise 2013-05-15, 18:05
liuyongbo 2013-05-16, 08:06