Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> how to print the channel capacity


Copy link to this message
-
RE: how to print the channel capacity
There are a few ways to monitor flume in operation. We use the JSON reporting, which is available via 'http://<agent address>:<port>/metrics'. You need to start the agent with the following parameters to get this interface:

-Dflume.monitoring.type=http -Dflume.monitoring.port=34545

We use cacti to graph channel size both as a percentage of maximum and absolute number of events in channel. This provides warning if the sinks cannot keep up with the sources.

We also graph ingress/egress event counts, much like a network bandwidth graph, for some channels to get an idea of the throughput and to see if sources/sinks are running at same speed.
________________________________
From: liuyongbo [mailto:[EMAIL PROTECTED]]
Sent: Tuesday, May 14, 2013 10:38 PM
To: [EMAIL PROTECTED]
Subject: how to print the channel capacity

Hi:
         I'm using flume to pass log data to mongodb, but I find that some data lose when the pressure is in high level, so I want to know the max request that flume can hold and need to print the capacity.but I can not find the proper way to do this instead of change the source code. Any ideas?
         thanks
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB