Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Thread Safety of KafkaStreams

Copy link to this message
Re: Thread Safety of KafkaStreams

One KafkaStream is meant to be iterated by a single thread. A better
approach is to request higher number of streams
from the Kafka consumer and let each process have its own KafkaStream.


On Tue, Nov 20, 2012 at 9:40 PM, David Ross <[EMAIL PROTECTED]> wrote:
> Hello,
> We want to process messages from a single KafkaStream in a number of
> processes. Is it possible to have this code executing in multiple threads
> against the same stream?
> for (message <- stream) {
>   someBlockingOperation(message)
> }
> The scaladocs mention thread safety, but some of the code seems fairly
> stateful. I was wondering if anyone has experience with this or knows if it
> will work?
> Thanks,
> David