Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Invalid KafkaStreams as a result of Decoder problems


Copy link to this message
-
Re: Invalid KafkaStreams as a result of Decoder problems
Neha Narkhede 2012-11-13, 17:27
Bob/Jason,

This is a bug. Ideally, we should allow restarting the consumer
streams when they encounter an error.
Can you file a JIRA ?

Thanks,
Neha

On Mon, Nov 12, 2012 at 6:31 AM, Bob Cotton <[EMAIL PROTECTED]> wrote:
> Handling the exceptions in the Decoder is not a problem.
>
> It was just a larger question about what to do in the event the stream gets
> into a failed state.
>
> Thanks
> - Bob
>
>
> On Sun, Nov 11, 2012 at 11:01 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
>> Currently, the code in Kafka doesn't expect the decoder to throw
>> exceptions. Could you write your decoder to return null on exceptions?
>>
>> Thanks,
>>
>> Jun
>>
>>  On Fri, Nov 9, 2012 at 8:17 AM, Bob Cotton <[EMAIL PROTECTED]> wrote:
>>
>>> During the implementation of a custom Encoder/Decoder we noticed that
>>> should the Decoder throw an exception, the KafkaStream that it is in use
>>> becomes invalid.
>>>
>>> Searching the mailing list indicates that the only way to recover from an
>>> invalid stream is to shutdown the whole high-level consumer and restart.
>>>
>>> Is there a better way to recover from this?
>>>
>>> Thanks for the responsiveness on the mailing list.
>>> - Bob
>>>
>>
>>