Kafka, mail # user - Re: partition and consumer stream number - 2012-12-19, 16:42
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: partition and consumer stream number
Thanks Jun, I didn't get you previous answer.
I think I didn't state my question clearly.

I know the number of the consumer in one consumer group should be less or equal than partition number.
But my question is about the relation between the stream number in one consumer and partition number subscribed by this consumer.  

For example this piece of code create only one stream in the consumer, so all partitions data will go into one stream?

    Map<String, Integer> topicCountMap = new HashMap<String, Integer>();
    topicCountMap.put(topic, new Integer(1));
    Map<String, List<KafkaStream<byte[], byte[]>>> consumerMap = consumer.createMessageStreams(topicCountMap);
    KafkaStream<byte[], byte[]> stream = consumerMap.get(topic).get(0);
    ConsumerIterator<byte[], byte[]> it = stream.iterator();
 
Sent from my iPad

ÔÚ 2012-12-19£¬23:49£¬Jun Rao <[EMAIL PROTECTED]> дµÀ£º

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB