Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> [0.6] Consumer API Question - What if a ConsumerIterator has an error?


Copy link to this message
-
Re: [0.6] Consumer API Question - What if a ConsumerIterator has an error?
Another problem - just taking toString of an iterator in a failed state
throws an exception.

On Thu, Dec 22, 2011 at 1:40 AM, David Ross <[EMAIL PROTECTED]> wrote:

> Howdy,
>
> I have come across a potential issue with the Consumer API in Kafka 0.6.
> After some tests and reading through the code, I believe the following
> scenario could result in an unhappy situation:
>
>    1. Create 5 streams listening to a topic
>    2. One of the streams times out (or has some other error) and it's
>    iterator is now in mode FAILED
>    3. (I think) Messages will now come into all five streams, but one of
>    them won't yield any new messages because of (2)
>
> The only solution I can think of is if an error is detected in any of the
> streams, all of them should be stopped and a new set of streams should be
> made. This seems unfavorable and would require messy concurrent programming.
>
> So is this an actual problem? Is there a better workaround? Have I
> misinterpreted something?
>
>
> Thanks,
>
> David
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB