Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Flume Monitoring Context


+
Matt.Elliott@... 2013-03-15, 11:54
Copy link to this message
-
Re: Flume Monitoring Context
Roshan Naik 2013-03-15, 18:26
Based on mem channel code.. should be similar semantics on file channel...

EventPutAttemptCount : Number of attempts to put events into the
channel. Updated when Source calls Transaction.put(event)

Channel Size - Number of events held in the channel currently

EventTakeSuccessCount : Number of take attempts by sink which actually
succeeded (a non null event was obtained). It is updated when
Transaction.take() is invoked.

EventTakeAttemptCount : Number of times an attempt to Take an event
form channel was made. It is updated when sink commits transaction.

Channel Capacity : The max  number of events that the channel is
configured to hold.
If  EventTakeAttemptCount - EventTakeSuccessCount  < batchSize on sink
side  it is usually an indication that an uncommitted transaction is
in flight on the sink side.

If  EventTakeAttemptCount - EventTakeSuccessCount  > batchSize and you
have only one sink attached to the channel, then the drain rate is
probably faster than the ingest rate. So in a few instance the
Transaction.take() yielded null to the sink.
Changes in ChannelSize is worth keeping an eye on. Increase in channel
size indicates a surge in ingest rate. When it decreases it indicates
a drop in ingest rate.

-Roshan
On Fri, Mar 15, 2013 at 4:54 AM,  <[EMAIL PROTECTED]> wrote:
> Can anyone shed some light on what each of these metrics are trying to
> represent? Some are self explanatory but I’m trying to figure out how I can
> use these to troubleshoot my deployment.
>
>
>
> {
>
> "CHANNEL.fileChannel":{"EventPutSuccessCount":"468085",
>
>                       "Type":"CHANNEL",
>
>                       "StopTime":"0",
>
>                       "EventPutAttemptCount":"468086",
>
>                       "ChannelSize":"233428",
>
>                       "StartTime":"1344882233070",
>
>                       "EventTakeSuccessCount":"458200",
>
>                       "ChannelCapacity":"600000",
>
>                       "EventTakeAttemptCount":"458288"},
>
> "CHANNEL.memChannel":{"EventPutSuccessCount":"22948908",
>
>                    "Type":"CHANNEL",
>
>                    "StopTime":"0",
>
>                    "EventPutAttemptCount":"22948908",
>
>                    "ChannelSize":"5",
>
>                    "StartTime":"1344882209413",
>
>                    "EventTakeSuccessCount":"22948900",
>
>                    "ChannelCapacity":"100",
>
>                    "EventTakeAttemptCount":"22948908"}
>
> }
>
>
>
> Thanks,
>
>
>
> Matt
>
> This message and/or attachments may include information subject to GDC4S
> S.P. 1.8.6 and GD Corporate Policy 07-105 and are intended to be accessed
> only by authorized recipients.  Use, storage and transmission are governed
> by General Dynamics and its policies. Contractual restrictions apply to
> third parties.  Recipients should refer to the policies or contract to
> determine proper handling.  Unauthorized review, use, disclosure or
> distribution is prohibited.  If you are not an intended recipient, please
> contact the sender and destroy all copies of the original message.
>
>