Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Thread Safety of KafkaStreams


Copy link to this message
-
Re: Thread Safety of KafkaStreams
David,

One KafkaStream is meant to be iterated by a single thread. A better
approach is to request higher number of streams
from the Kafka consumer and let each process have its own KafkaStream.

Thanks,
Neha

On Tue, Nov 20, 2012 at 9:40 PM, David Ross <[EMAIL PROTECTED]> wrote:
> Hello,
>
> We want to process messages from a single KafkaStream in a number of
> processes. Is it possible to have this code executing in multiple threads
> against the same stream?
>
> for (message <- stream) {
>   someBlockingOperation(message)
> }
>
> The scaladocs mention thread safety, but some of the code seems fairly
> stateful. I was wondering if anyone has experience with this or knows if it
> will work?
>
>
> Thanks,
>
> David
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB