Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Getting fixed amount of messages using Zookeeper based consumer


Copy link to this message
-
Getting fixed amount of messages using Zookeeper based consumer
Vaibhav Puranik 2012-07-12, 00:05
Hi all,

Is there any way to get a fixed amount of messages using Zookeeper based
consumer (ConsumerConnector)?

I know that with SimpleConsumer you can pass fetchSize as an argument and
limit the number of messages coming back.

This sample code creates 4 threads that keep consuming forever.

// consume the messages in the threads
for(final KafkaStream<Message> stream: streams) {
  executor.submit(new Runnable() {
    public void run() {
      for(MessageAndMetadata msgAndMetadata: stream) {
        // process message (msgAndMetadata.message())
      }
    }
  });
}

Regards,
Vaibhav