Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Invalid KafkaStreams as a result of Decoder problems


Copy link to this message
-
Re: Invalid KafkaStreams as a result of Decoder problems
Bob/Jason,

This is a bug. Ideally, we should allow restarting the consumer
streams when they encounter an error.
Can you file a JIRA ?

Thanks,
Neha

On Mon, Nov 12, 2012 at 6:31 AM, Bob Cotton <[EMAIL PROTECTED]> wrote:
> Handling the exceptions in the Decoder is not a problem.
>
> It was just a larger question about what to do in the event the stream gets
> into a failed state.
>
> Thanks
> - Bob
>
>
> On Sun, Nov 11, 2012 at 11:01 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
>> Currently, the code in Kafka doesn't expect the decoder to throw
>> exceptions. Could you write your decoder to return null on exceptions?
>>
>> Thanks,
>>
>> Jun
>>
>>  On Fri, Nov 9, 2012 at 8:17 AM, Bob Cotton <[EMAIL PROTECTED]> wrote:
>>
>>> During the implementation of a custom Encoder/Decoder we noticed that
>>> should the Decoder throw an exception, the KafkaStream that it is in use
>>> becomes invalid.
>>>
>>> Searching the mailing list indicates that the only way to recover from an
>>> invalid stream is to shutdown the whole high-level consumer and restart.
>>>
>>> Is there a better way to recover from this?
>>>
>>> Thanks for the responsiveness on the mailing list.
>>> - Bob
>>>
>>
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB