Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Best way to start / stop / restart a Consumer?


Copy link to this message
-
Re: Best way to start / stop / restart a Consumer?
Each time we create a new consumer connector, we assign a random consumer
id by default. You can try setting "consumer.id" to use a fixed consumer
id. In any case, we probably should deregister those beans when shutting
down the connector. Could you file a jira?

Thanks,

Jun
On Thu, Oct 10, 2013 at 7:55 AM, Tanguy tlrx <[EMAIL PROTECTED]> wrote:

> It's in 0.8.
>
> The JMX names are not exactly the same, but I see 2 beans with similar
> names, something like:
>
>
> "my_consumer_group-my_consumer-mytopic-my_consumer_hostname.thing.com-1381416134138-3573c8bf-7-FetchQueueSize"
>
> -- Tanguy
>
>
>
>
>
> 2013/10/10 Jun Rao <[EMAIL PROTECTED]>
>
> > Is that in 0.7 or 0.8? JMX won't allow a bean with the same name to be
> > registered twice. Do you see 2 beans with similar names? What are the
> exact
> > bean names?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Oct 10, 2013 at 2:42 AM, Tanguy tlrx <[EMAIL PROTECTED]> wrote:
> >
> > > Hi all,
> > >
> > > In our application, we are currently starting a Kafka Consumer with the
> > > following lines of code:
> > >
> > > connector = Consumer.createJavaConsumerConnector(consumerConfig);
> > > streams = connector .createMessageStreams(map);
> > >
> > > Then, each KafkaStream is processed in a dedicated thread per topic and
> > > partition, as documented here
> > >
> https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Group+Example
> > >
> > > We need to stop (halt) and restart the consumer. Today, we just call:
> > >
> > > connector.shutdown()
> > >
> > > and wait for threads to terminate.
> > >
> > > To restart the consumer, we create a new connector:
> > >
> > > connector = Consumer.createJavaConsumerConnector(consumerConfig);
> > >
> > > When restarting is complete, I can see that a JMX MBean (we  use
> Metrics
> > > JMXReporter) like "ZookeeperConsumerConnector" is registered twice.
> This
> > > bean is not registered when the previous connector instance is shut
> down.
> > >
> > > What is the best way to stop/halt and restart a Consumer using the Java
> > > API?
> > >
> > > Is it normal that the MBean is not unregistered at shutdown time?
> > >
> > > Thanks,
> > >
> > > -- Tanguy
> > >
> >
>
>
>
> --
> -- Tanguy
> twitter @tlrx
> https://github.com/tlrx
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB