Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Broker problem


+
Bob Jervis 2013-03-08, 19:56
Copy link to this message
-
Re: Broker problem
Neha Narkhede 2013-03-09, 23:02
Bob,

This seems to indicate that the broker is killing the connection to the
consumer. I think the broker logs should have some errors explaining the
issue. Please can you file a JIRA, explain the set of steps you follow to
reproduce the issue and also attach the broker and consumer logs there ?

Thanks,
Neha
On Fri, Mar 8, 2013 at 10:51 AM, Bob Jervis <[EMAIL PROTECTED]> wrote:

> I am re-sending this thread from my personal account because my company
> e-mail (visibletechnologies.com) is not receiving replies from yur mailing
> list.
>
>
> To recap, I was seeing a steady stream of exceptions both from the broker
> and a client.  Neha recommended setting ulimit.  Our machines were at the
> default of 1024.  Setting the limit to 65536 dramatically reduced the
> broker-side exception spew.
>
> However, repeating the tests from yesterday, I am still seeing the
> following repeated block of log entries on our client:
>
> 2013-03-08 18:44:41,063 INFO kafka.consumer.SimpleConsumer: Reconnect due
> to socket error:
> java.io.EOFException: Received -1 when reading from channel, socket has
> likely been closed.
>         at kafka.utils.Utils$.read(Utils.scala:373)
>         at
>
> kafka.network.BoundedByteBufferReceive.readFrom(BoundedByteBufferReceive.scala:67)
>         at
> kafka.network.Receive$class.readCompletely(Transmission.scala:56)
>         at
>
> kafka.network.BoundedByteBufferReceive.readCompletely(BoundedByteBufferReceive.scala:29)
>         at kafka.network.BlockingChannel.receive(BlockingChannel.scala:100)
>         at
> kafka.consumer.SimpleConsumer.liftedTree1$1(SimpleConsumer.scala:124)
>         at
>
> kafka.consumer.SimpleConsumer.kafka$consumer$SimpleConsumer$$sendRequest(SimpleConsumer.scala:122)
>         at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply$mcV$sp(SimpleConsumer.scala:161)
>         at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:161)
>         at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1$$anonfun$apply$mcV$sp$1.apply(SimpleConsumer.scala:161)
>         at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
>         at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply$mcV$sp(SimpleConsumer.scala:160)
>         at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:160)
>         at
>
> kafka.consumer.SimpleConsumer$$anonfun$fetch$1.apply(SimpleConsumer.scala:160)
>         at kafka.metrics.KafkaTimer.time(KafkaTimer.scala:33)
>         at kafka.consumer.SimpleConsumer.fetch(SimpleConsumer.scala:159)
>         at
> kafka.server.AbstractFetcherThread.doWork(AbstractFetcherThread.scala:93)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:50)
> 2013-03-08 18:44:41,391 INFO kafka.consumer.ConsumerFetcherManager:
> [ConsumerFetcherManager-1362767252595] removing fetcher on topic
> VTFull-enriched, partition 0
> 2013-03-08 18:44:41,392 INFO kafka.utils.VerifiableProperties: Verifying
> properties
> 2013-03-08 18:44:41,393 INFO kafka.utils.VerifiableProperties: Property
> broker.list is overridden to 10.10.2.123:9092
> 2013-03-08 18:44:41,393 INFO kafka.utils.VerifiableProperties: Property
> clientid is overridden to K-Router
> 2013-03-08 18:44:41,393 INFO kafka.client.ClientUtils$: Fetching metadata
> for topic Set(VTFull-enriched)
> 2013-03-08 18:44:41,393 INFO kafka.producer.SyncProducer: Connected to
> 10.10.2.123:9092 for producing
> 2013-03-08 18:44:41,402 INFO kafka.producer.SyncProducer: Disconnecting
> from 10.10.2.123:9092
> 2013-03-08 18:44:41,402 INFO kafka.consumer.ConsumerFetcherManager:
> [ConsumerFetcherManager-1362767252595] adding fetcher on topic
> VTFull-enriched, partion 0, initOffset 14935 to broker 0 with fetcherId 0
>
>
> We are currently using the high-level consumer.
>
> Jun's question about java 5 bugs: we are using Java 1.6, so I would assume
> we have that fix.  I will, however,folllow up and see if we can confirm
> whether this bug is biting us.