Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Review Request: FLUME-1940, FLUME-1957 - Logging Snapshots of Flume Metrics on Shutdown


+
Israel Ekpo 2013-04-11, 04:44
+
Mike Percy 2013-04-16, 07:10
Copy link to this message
-
Re: Review Request: FLUME-1940, FLUME-1957 - Logging Snapshots of Flume Metrics on Shutdown


> On April 16, 2013, 7:10 a.m., Mike Percy wrote:
> > Looks good overall. However I think there is an issue where there are certain assumptions made about keys existing when they may not be set.
> > I am getting a NullPointerException at stop() time (when I kill Flume via Ctrl-C) when running with this config file:
> >
> > agent.sources = seqGenSrc
> > agent.channels = memoryChannel
> > agent.sinks = nullSink
> >
> > agent.channels.memoryChannel.type = memory
> > agent.channels.memoryChannel.capacity = 100
> >
> > agent.sources.seqGenSrc.type = seq
> > agent.sources.seqGenSrc.channels = memoryChannel
> >
> > agent.sinks.nullSink.type = null
> > agent.sinks.nullSink.channel = memoryChannel
> >
> > I tweaked the patch a little to validate (basically using Preconditions.checkNotNull(counterMap.get(counter), ...)) to get a helpful error message for this and I see the following:
> >
> > Exception in thread "agent-shutdown-hook" java.lang.NullPointerException: Unknown counter: channel.start.time
> > at com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> > at org.apache.flume.instrumentation.MonitoredCounterGroup.get(MonitoredCounterGroup.java:192)
> > at org.apache.flume.instrumentation.MonitoredCounterGroup.stop(MonitoredCounterGroup.java:153)
> > at org.apache.flume.channel.MemoryChannel.stop(MemoryChannel.java:355)
> > at org.apache.flume.lifecycle.LifecycleSupervisor.stop(LifecycleSupervisor.java:106)
> > at org.apache.flume.node.Application.stop(Application.java:92)
> > at org.apache.flume.node.Application$1.run(Application.java:302)

I will be posting an updated patch soon.

channel.start.time and channel.stop.time are artificial keys I just added to a copy of the orignal map.
- Israel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10416/#review19246
-----------------------------------------------------------
On April 11, 2013, 4:44 a.m., Israel Ekpo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10416/
> -----------------------------------------------------------
>
> (Updated April 11, 2013, 4:44 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Logging Metrics from the stop() method for the following components: ChannelCounter, ChannelProcessorCounter, SinkCounter, SinkProcessorCounter, SourceCounter
>
>
> This addresses bugs FLUME-1940 and FLUME-1957.
>     https://issues.apache.org/jira/browse/FLUME-1940
>     https://issues.apache.org/jira/browse/FLUME-1957
>
>
> Diffs
> -----
>
>   flume-ng-core/src/main/java/org/apache/flume/instrumentation/MonitoredCounterGroup.java 502fe9e
>
> Diff: https://reviews.apache.org/r/10416/diff/
>
>
> Testing
> -------
>
>
> Thanks,
>
> Israel Ekpo
>
>

+
Israel Ekpo 2013-04-16, 14:58
+
Israel Ekpo 2013-04-23, 04:21
+
Israel Ekpo 2013-04-24, 11:16
+
Mike Percy 2013-04-26, 08:52