Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> auto.offset.reset in kafka 0.8


+
Martin Eigenbrodt 2013-07-01, 07:51
+
Jun Rao 2013-07-01, 15:45
Copy link to this message
-
Re: auto.offset.reset in kafka 0.8

On 01.07.2013, at 17:45, Jun Rao <[EMAIL PROTECTED]> wrote:
> Not sure that I fully understand your problem. Could you attach the
> exception that you saw?

Here is what I do and get:

I use the High Level Consumer and configured it with

auto.offset.reset = other
 
I get

kafka.common.InvalidConfigException: Wrong value other of auto.offset.reset in ConsumerConfig; Valid values are smallest and largest
        at kafka.consumer.ConsumerConfig$.validateAutoOffsetReset(Unknown Source) ~[kafka-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT+37]
        at kafka.consumer.ConsumerConfig$.validate(Unknown Source) ~[kafka-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT+37]
        at kafka.consumer.ConsumerConfig.<init>(Unknown Source) ~[kafka-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT+37]
        at kafka.consumer.ConsumerConfig.<init>(Unknown Source) ~[kafka-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT+37]

as soon as I try to  

    val consumerConfig = new ConsumerConfig(props);

What I expected:

 - No Exception on initialization
 - No Exception if the consumer can connect to Zookeeper and fetch a valid offset
 - An Exception if it can not reach Zookeeper or it does not contain a valid offset for the given consumer group
Regards,

Martin

>
> Thanks,
>
> Jun
>
>
> On Mon, Jul 1, 2013 at 12:50 AM, Martin Eigenbrodt <
> [EMAIL PROTECTED]> wrote:
>
>> Hi there,
>>
>> I am using kafka 0.8 (currently 0.8.0-beta1-candidate1) and I want my
>> consumer to fail if it can not reliably find out the last consumed offset
>> from zookeeper.
>>
>> According to https://kafka.apache.org/08/configuration.html for "
>> https://kafka.apache.org/08/configuration.html"
>>
>>> What to do when there is no initial offset in Zookeeper or if an offset
>> is out of range:
>> ..
>>> * anything else: throw exception to the consumer
>>
>>
>> But it looks like the exception is always thrown, not only when there is
>> no initial offset found in Zookeper. It;s being validated at startup...
>> Is there a workaround to fail only if no offset is found in zk?
>>
>> Regards,
>>
>> Martin
 
+
Jun Rao 2013-07-02, 14:37