Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume Monitoring Context


Copy link to this message
-
Re: Flume Monitoring Context
Based on mem channel code.. should be similar semantics on file channel...

EventPutAttemptCount : Number of attempts to put events into the
channel. Updated when Source calls Transaction.put(event)

Channel Size - Number of events held in the channel currently

EventTakeSuccessCount : Number of take attempts by sink which actually
succeeded (a non null event was obtained). It is updated when
Transaction.take() is invoked.

EventTakeAttemptCount : Number of times an attempt to Take an event
form channel was made. It is updated when sink commits transaction.

Channel Capacity : The max  number of events that the channel is
configured to hold.
If  EventTakeAttemptCount - EventTakeSuccessCount  < batchSize on sink
side  it is usually an indication that an uncommitted transaction is
in flight on the sink side.

If  EventTakeAttemptCount - EventTakeSuccessCount  > batchSize and you
have only one sink attached to the channel, then the drain rate is
probably faster than the ingest rate. So in a few instance the
Transaction.take() yielded null to the sink.
Changes in ChannelSize is worth keeping an eye on. Increase in channel
size indicates a surge in ingest rate. When it decreases it indicates
a drop in ingest rate.

-Roshan
On Fri, Mar 15, 2013 at 4:54 AM,  <[EMAIL PROTECTED]> wrote:
> Can anyone shed some light on what each of these metrics are trying to
> represent? Some are self explanatory but I’m trying to figure out how I can
> use these to troubleshoot my deployment.
>
>
>
> {
>
> "CHANNEL.fileChannel":{"EventPutSuccessCount":"468085",
>
>                       "Type":"CHANNEL",
>
>                       "StopTime":"0",
>
>                       "EventPutAttemptCount":"468086",
>
>                       "ChannelSize":"233428",
>
>                       "StartTime":"1344882233070",
>
>                       "EventTakeSuccessCount":"458200",
>
>                       "ChannelCapacity":"600000",
>
>                       "EventTakeAttemptCount":"458288"},
>
> "CHANNEL.memChannel":{"EventPutSuccessCount":"22948908",
>
>                    "Type":"CHANNEL",
>
>                    "StopTime":"0",
>
>                    "EventPutAttemptCount":"22948908",
>
>                    "ChannelSize":"5",
>
>                    "StartTime":"1344882209413",
>
>                    "EventTakeSuccessCount":"22948900",
>
>                    "ChannelCapacity":"100",
>
>                    "EventTakeAttemptCount":"22948908"}
>
> }
>
>
>
> Thanks,
>
>
>
> Matt
>
> This message and/or attachments may include information subject to GDC4S
> S.P. 1.8.6 and GD Corporate Policy 07-105 and are intended to be accessed
> only by authorized recipients.  Use, storage and transmission are governed
> by General Dynamics and its policies. Contractual restrictions apply to
> third parties.  Recipients should refer to the policies or contract to
> determine proper handling.  Unauthorized review, use, disclosure or
> distribution is prohibited.  If you are not an intended recipient, please
> contact the sender and destroy all copies of the original message.
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB