Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Shutdown metric question


+
DSuiter RDX 2013-10-08, 19:03
Copy link to this message
-
Re: Shutdown metric question
Hari Shreedharan 2013-10-08, 22:20
No, it does not mean your channel dropped data. It just means that the sink tried to read data 695 times from the channel, but the channel could only return 471 - obviously because only that many events were written in the first place.
Thanks,
Hari
On Tuesday, October 8, 2013 at 12:03 PM, DSuiter RDX wrote:

> Hi all,
>
> We have a TCP source channeled to an Avro sink, which is connecting to an Avro source channeled to HDFS sink. Everything is flowing, but I noticed this when it shut down:
>
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel. channel.event.put.attempt == 471
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel. channel.event.put.success == 471
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel. channel.event.take.attempt == 695
> Shutdown Metric for type: CHANNEL, name: memory_RT_Tier2_Channel. channel.event.take.success == 471
>
> Does the difference between the channel.take.attempt == 695 and take.success == 471 indicate that my channel dropped data, or does it indicate that the channel tried to grab from the source but did not get data from it? There is a period when the Avro RPC call connects where it is not fully socketed, so maybe the channel is grabbing air? If so, that is ok. If it indicates data got lost, that is not acceptable and hopefully someone can help me correct the root cause there.
>
> Thanks!
> Devin Suiter
> Jr. Data Solutions Software Engineer
>
> 100 Sandusky Street | 2nd Floor | Pittsburgh, PA 15212
> Google Voice: 412-256-8556 | www.rdx.com (http://www.rdx.com/)

+
DSuiter RDX 2013-10-09, 12:40