Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> auto.offset.reset in kafka 0.8


Copy link to this message
-
Re: auto.offset.reset in kafka 0.8

On 01.07.2013, at 17:45, Jun Rao <[EMAIL PROTECTED]> wrote:
> Not sure that I fully understand your problem. Could you attach the
> exception that you saw?

Here is what I do and get:

I use the High Level Consumer and configured it with

auto.offset.reset = other
 
I get

kafka.common.InvalidConfigException: Wrong value other of auto.offset.reset in ConsumerConfig; Valid values are smallest and largest
        at kafka.consumer.ConsumerConfig$.validateAutoOffsetReset(Unknown Source) ~[kafka-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT+37]
        at kafka.consumer.ConsumerConfig$.validate(Unknown Source) ~[kafka-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT+37]
        at kafka.consumer.ConsumerConfig.<init>(Unknown Source) ~[kafka-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT+37]
        at kafka.consumer.ConsumerConfig.<init>(Unknown Source) ~[kafka-0.8.0-SNAPSHOT.jar:0.8.0-SNAPSHOT+37]

as soon as I try to  

    val consumerConfig = new ConsumerConfig(props);

What I expected:

 - No Exception on initialization
 - No Exception if the consumer can connect to Zookeeper and fetch a valid offset
 - An Exception if it can not reach Zookeeper or it does not contain a valid offset for the given consumer group
Regards,

Martin

>
> Thanks,
>
> Jun
>
>
> On Mon, Jul 1, 2013 at 12:50 AM, Martin Eigenbrodt <
> [EMAIL PROTECTED]> wrote:
>
>> Hi there,
>>
>> I am using kafka 0.8 (currently 0.8.0-beta1-candidate1) and I want my
>> consumer to fail if it can not reliably find out the last consumed offset
>> from zookeeper.
>>
>> According to https://kafka.apache.org/08/configuration.html for "
>> https://kafka.apache.org/08/configuration.html"
>>
>>> What to do when there is no initial offset in Zookeeper or if an offset
>> is out of range:
>> ..
>>> * anything else: throw exception to the consumer
>>
>>
>> But it looks like the exception is always thrown, not only when there is
>> no initial offset found in Zookeper. It;s being validated at startup...
>> Is there a workaround to fail only if no offset is found in zk?
>>
>> Regards,
>>
>> Martin
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB