Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> [0.6] Consumer API Question - What if a ConsumerIterator has an error?

David Ross 2011-12-22, 09:40
Neha Narkhede 2011-12-22, 18:10
Copy link to this message
Re: [0.6] Consumer API Question - What if a ConsumerIterator has an error?
Another problem - just taking toString of an iterator in a failed state
throws an exception.

On Thu, Dec 22, 2011 at 1:40 AM, David Ross <[EMAIL PROTECTED]> wrote:

> Howdy,
> I have come across a potential issue with the Consumer API in Kafka 0.6.
> After some tests and reading through the code, I believe the following
> scenario could result in an unhappy situation:
>    1. Create 5 streams listening to a topic
>    2. One of the streams times out (or has some other error) and it's
>    iterator is now in mode FAILED
>    3. (I think) Messages will now come into all five streams, but one of
>    them won't yield any new messages because of (2)
> The only solution I can think of is if an error is detected in any of the
> streams, all of them should be stopped and a new set of streams should be
> made. This seems unfavorable and would require messy concurrent programming.
> So is this an actual problem? Is there a better workaround? Have I
> misinterpreted something?
> Thanks,
> David
Neha Narkhede 2012-01-05, 01:55