Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Error messages in logs


Copy link to this message
-
Re: Error messages in logs
Joel Koshy 2013-08-12, 17:42
The closed by interrupt exception is because the underlying
"leader-finder-thread" has been interrupted - this is done for e.g.,
on a rebalance. Do you see any rebalancing messages in your consumer
logs? How many topics are you consuming? Are the topics present on the
broker and is there a leader for each of them (i.e., without any
leader movement)? Can you try running a console consumer to see if you
see the same behavior?

Thanks,

Joel
On Mon, Aug 12, 2013 at 10:27 AM, Eric Sites <[EMAIL PROTECTED]> wrote:
> Very clean logs.
>
> Just deletes and rolls messages:
>
> [2013-08-12 16:49:15,371] INFO [Kafka Log on Broker 4], Rolling incoming-6
> due to full data log (kafka.log.Log)
> [2013-08-12 16:49:15,429] INFO [Kafka Log on Broker 4], Rolling log
> 'incoming-6' to /mnt/data/kafka/incoming-6/00000000000000434346.log and
> /mnt/data/kafka/incoming-6/00000000000000434346.index (kafka.log.Log)
>
> [2013-08-12 17:18:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 63488 from incoming-1 (kafka.log.Log)
> [2013-08-12 17:18:59,636] INFO Deleting index
> /mnt/data/kafka/incoming-1/00000000000000063488.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:19:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 11276 from incoming-4 (kafka.log.Log)
> [2013-08-12 17:19:59,635] INFO Deleting index
> /mnt/data/kafka/incoming-4/00000000000000011276.index
> (kafka.log.OffsetIndex)
> [2013-08-12 17:19:59,636] INFO [Kafka Log on Broker 4], Deleting log
> segment 63924 from incoming-6 (kafka.log.Log)
> [2013-08-12 17:19:59,637] INFO Deleting index
> /mnt/data/kafka/incoming-6/00000000000000063924.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:19:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 11276 from incoming-4 (kafka.log.Log)
> [2013-08-12 17:19:59,635] INFO Deleting index
> /mnt/data/kafka/incoming-4/00000000000000011276.index
> (kafka.log.OffsetIndex)
> [2013-08-12 17:19:59,636] INFO [Kafka Log on Broker 4], Deleting log
> segment 63924 from incoming-6 (kafka.log.Log)
> [2013-08-12 17:19:59,637] INFO Deleting index
> /mnt/data/kafka/incoming-6/00000000000000063924.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:20:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 64723 from incoming-0 (kafka.log.Log)
> [2013-08-12 17:20:59,636] INFO Deleting index
> /mnt/data/kafka/incoming-0/00000000000000064723.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:20:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 64723 from incoming-0 (kafka.log.Log)
> [2013-08-12 17:20:59,636] INFO Deleting index
> /mnt/data/kafka/incoming-0/00000000000000064723.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:23:59,636] INFO [Kafka Log on Broker 4], Deleting log
> segment 65478 from incoming-1 (kafka.log.Log)
> [2013-08-12 17:23:59,637] INFO Deleting index
> /mnt/data/kafka/incoming-1/00000000000000065478.index
> (kafka.log.OffsetIndex)
>
>
> Cheers,
> Eric Sites
>
> On 8/12/13 1:20 PM, "Joel Koshy" <[EMAIL PROTECTED]> wrote:
>
>>The consumer uses a sync.producer to look up topic metadata for topics
>>that it consumes. For some reason, it is unable to establish a
>>connection to the brokers. Are the brokers all up with clean logs?
>>
>>On Mon, Aug 12, 2013 at 8:39 AM, Eric Sites <[EMAIL PROTECTED]>
>>wrote:
>>> I am getting some new messages in my 0.8 Java producer app.
>>>
>>> I am running 6 Kafka brokers.
>>>
>>> The app producing these logs messages is not doing anything, i.e. Not
>>>producing messages.
>>>
>>> Anywhere form every 5 minutes to 10 seconds sometimes.
>>>
>>> [2013-08-12 14:45:20,714] ERROR Producer connection to kafka002:9092
>>>unsuccessful (kafka.producer.SyncProducer)
>>> java.nio.channels.ClosedByInterruptException
>>> at
>>>java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterrupti
>>>bleChannel.java:184)
>>> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:543)
>>> at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
>>