Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Error when restarting consumer


Copy link to this message
-
Re: Error when restarting consumer
Jun Rao 2013-06-24, 05:09
Are you consuming more than 2GB of data for a single fetch request (for all
partitions)? If so, this can cause overflow in the request (response) size
since it's represented as an int. The solution is to reduce the fetch size.

Thanks,

Jun
On Sun, Jun 23, 2013 at 9:30 PM, anand nalya <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I'm using High Level Consumer with kafka 0.7.2. The consumer is able to
> consume messages from kafka for the first time but when I restart the
> consumer process, I'm getting the following error:
>
> 2013-06-24 09:48:25 FetchRunnable-0 [ERROR] FetcherRunnable:89 - error in
> FetcherRunnable
> kafka.network.InvalidRequestException: -2147400679 is not a valid request
> size.
>     at
>
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:60)
>     at kafka.network.Receive$class.readCompletely(Transmission.scala:55)
>     at
>
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
>     at kafka.consumer.SimpleConsumer.getResponse(SimpleConsumer.scala:177)
>     at
> kafka.consumer.SimpleConsumer.liftedTree2$1(SimpleConsumer.scala:117)
>     at kafka.consumer.SimpleConsumer.multifetch(SimpleConsumer.scala:115)
>     at kafka.consumer.FetcherRunnable.run(FetcherRunnable.scala:60)
>
> I'm not sure how is it getting -2147400679 as the request size. Any ideas
> or pointers what may be wrong here?
>
> Thanks,
> Anand
>