Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> createMessageStreamsByFilter unexpected behaviour


+
Michal Haris 2012-11-20, 12:44
Copy link to this message
-
Re: createMessageStreamsByFilter unexpected behaviour
This is likely caused by https://issues.apache.org/jira/browse/KAFKA-550.
The fix has been checked into trunk.

Thanks,

Jun

On Tue, Nov 20, 2012 at 4:44 AM, Michal Haris <[EMAIL PROTECTED]>wrote:

> Hi, I am seeing behaviour which I am not expecting when using topic
> filters.
>
> TopicFilter sourceTopicFilter = new Whitelist("pageviews");
> List<KafkaStream<Message>> streams > consumer.createMessageStreamsByFilter(sourceTopicFilter, 3);
>
> The topic has exactly 3 partitions and 3 streams are created, however only
> the last one gets messages, the first two become stale.
> I am not sure if the singe stream contains messages from all partitions or
> When I however use the following code I get all 3 streams consuming for
> each respective partition.
>
> Map<String, List<KafkaStream<Message>>> topicStreams > consumer.createMessageStreams(
>     new HashMap<String, Integer>() {{
>         put("pageviews", 3);
>     }}
> );
> List<KafkaStream<Message>> streams = topicStreams.get("pageviews");
>
> Am I missing something, is this expected ?
>
> --
> Michal Haris
> Software Engineer
>
> www.visualdna.com | t: +44 (0) 207 734 7033
>
+
Michal Haris 2012-11-21, 19:04
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB