Kafka, mail # user - Multiple partitions per thread behavior - 2014-05-01, 23:14
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Multiple partitions per thread behavior
Hey gang, quick consumer behavior question:

If I have a Consumer that owns several partitions but only creates a single stream for them, in what order will that stream get its messages? From my experiments it seems like the Consumer fetches some amount of messages for all N partitions it holds, and then processes all the messages for Stream 1, then Stream 2, etc. until it has run out of messages, then fetches more. Is that accurate?

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB