Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Re: Flume-ng InstanceAlreadyExistsException


Copy link to this message
-
Re: Flume-ng InstanceAlreadyExistsException
Hi Deepesh: I think you can ignore the exception stack, its benign. Flume
tries to re-register the monitoring MBean whenever the configurations are
dynamically refreshed and it re-initializes the components. And because the
MBean was already registered, you see this exception trace. I remember
there was a Jira ticket to suppress this and Mike/Hari fixed the same in
trunk.

On the other question I assume you are either asking if there a minimum of
the number sinks required or if there is limit to the total number of sinks
in an instance, for both questions the answer is NO.

I would think of the heap size proportional to the Channel's (Memory) size
you are setting up, to make sure there is enough space to hold all the
events when required. The number of sinks that are setup, would relate to
the number of threads (/cores) available for you to use.

Other folks can chime in and add as required.

Thanks
Venkatesh

On Wed, Sep 4, 2013 at 11:54 PM, Deepesh Malviya <[EMAIL PROTECTED]> wrote:

> Hi,
>
> We have configured flume-ng in our environment & we dynamically update the
> configuration. In our case, we will be having 20+ sinks & will increase as
> we update the conf. The problem we are seeing in it is the following
> exception is coming repeatedly for most of the configured sinks
>
>
> 05 Sep 2013 05:00:38,770 ERROR [lifecycleSupervisor-1-3]
> (org.apache.flume.instrumentation.MonitoredCounterGroup.register:113)  -
> Failed to register monitored counter group for type: SINK, name: serviceSink
> javax.management.InstanceAlreadyExistsException:
> org.apache.flume.sink:type=serviceSink
> at com.sun.jmx.mbeanserver.Repository.addMBean(Repository.java:467)
>  at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.internal_addObject(DefaultMBeanServerInterceptor.java:1520)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerDynamicMBean(DefaultMBeanServerInterceptor.java:986)
>  at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerObject(DefaultMBeanServerInterceptor.java:938)
> at
> com.sun.jmx.interceptor.DefaultMBeanServerInterceptor.registerMBean(DefaultMBeanServerInterceptor.java:330)
>  at
> com.sun.jmx.mbeanserver.JmxMBeanServer.registerMBean(JmxMBeanServer.java:517)
> at
> org.apache.flume.instrumentation.MonitoredCounterGroup.register(MonitoredCounterGroup.java:108)
>  at
> org.apache.flume.instrumentation.MonitoredCounterGroup.start(MonitoredCounterGroup.java:88)
> at org.apache.flume.sink.hdfs.HDFSEventSink.start(HDFSEventSink.java:484)
>  at
> org.apache.flume.sink.DefaultSinkProcessor.start(DefaultSinkProcessor.java:46)
> at org.apache.flume.SinkRunner.start(SinkRunner.java:79)
>  at
> org.apache.flume.lifecycle.LifecycleSupervisor$MonitorRunnable.run(LifecycleSupervisor.java:251)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
>  at
> java.util.concurrent.FutureTask$Sync.innerRunAndReset(FutureTask.java:351)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:178)
>  at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:165)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:267)
>  at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1146)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
>  at java.lang.Thread.run(Thread.java:679)
>
>
> What is the cause of this error?
> Also, is there any limited number of sinks that should be configured with
> one flume-ng instance. The reason for asking this particular question is
> because of the heap size we configure in flume-ng.
>
>
> --
> _Deepesh
>