Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Error messages in logs


Copy link to this message
-
Re: Error messages in logs
The closed by interrupt exception is because the underlying
"leader-finder-thread" has been interrupted - this is done for e.g.,
on a rebalance. Do you see any rebalancing messages in your consumer
logs? How many topics are you consuming? Are the topics present on the
broker and is there a leader for each of them (i.e., without any
leader movement)? Can you try running a console consumer to see if you
see the same behavior?

Thanks,

Joel
On Mon, Aug 12, 2013 at 10:27 AM, Eric Sites <[EMAIL PROTECTED]> wrote:
> Very clean logs.
>
> Just deletes and rolls messages:
>
> [2013-08-12 16:49:15,371] INFO [Kafka Log on Broker 4], Rolling incoming-6
> due to full data log (kafka.log.Log)
> [2013-08-12 16:49:15,429] INFO [Kafka Log on Broker 4], Rolling log
> 'incoming-6' to /mnt/data/kafka/incoming-6/00000000000000434346.log and
> /mnt/data/kafka/incoming-6/00000000000000434346.index (kafka.log.Log)
>
> [2013-08-12 17:18:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 63488 from incoming-1 (kafka.log.Log)
> [2013-08-12 17:18:59,636] INFO Deleting index
> /mnt/data/kafka/incoming-1/00000000000000063488.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:19:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 11276 from incoming-4 (kafka.log.Log)
> [2013-08-12 17:19:59,635] INFO Deleting index
> /mnt/data/kafka/incoming-4/00000000000000011276.index
> (kafka.log.OffsetIndex)
> [2013-08-12 17:19:59,636] INFO [Kafka Log on Broker 4], Deleting log
> segment 63924 from incoming-6 (kafka.log.Log)
> [2013-08-12 17:19:59,637] INFO Deleting index
> /mnt/data/kafka/incoming-6/00000000000000063924.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:19:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 11276 from incoming-4 (kafka.log.Log)
> [2013-08-12 17:19:59,635] INFO Deleting index
> /mnt/data/kafka/incoming-4/00000000000000011276.index
> (kafka.log.OffsetIndex)
> [2013-08-12 17:19:59,636] INFO [Kafka Log on Broker 4], Deleting log
> segment 63924 from incoming-6 (kafka.log.Log)
> [2013-08-12 17:19:59,637] INFO Deleting index
> /mnt/data/kafka/incoming-6/00000000000000063924.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:20:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 64723 from incoming-0 (kafka.log.Log)
> [2013-08-12 17:20:59,636] INFO Deleting index
> /mnt/data/kafka/incoming-0/00000000000000064723.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:20:59,635] INFO [Kafka Log on Broker 4], Deleting log
> segment 64723 from incoming-0 (kafka.log.Log)
> [2013-08-12 17:20:59,636] INFO Deleting index
> /mnt/data/kafka/incoming-0/00000000000000064723.index
> (kafka.log.OffsetIndex)
>
> [2013-08-12 17:23:59,636] INFO [Kafka Log on Broker 4], Deleting log
> segment 65478 from incoming-1 (kafka.log.Log)
> [2013-08-12 17:23:59,637] INFO Deleting index
> /mnt/data/kafka/incoming-1/00000000000000065478.index
> (kafka.log.OffsetIndex)
>
>
> Cheers,
> Eric Sites
>
> On 8/12/13 1:20 PM, "Joel Koshy" <[EMAIL PROTECTED]> wrote:
>
>>The consumer uses a sync.producer to look up topic metadata for topics
>>that it consumes. For some reason, it is unable to establish a
>>connection to the brokers. Are the brokers all up with clean logs?
>>
>>On Mon, Aug 12, 2013 at 8:39 AM, Eric Sites <[EMAIL PROTECTED]>
>>wrote:
>>> I am getting some new messages in my 0.8 Java producer app.
>>>
>>> I am running 6 Kafka brokers.
>>>
>>> The app producing these logs messages is not doing anything, i.e. Not
>>>producing messages.
>>>
>>> Anywhere form every 5 minutes to 10 seconds sometimes.
>>>
>>> [2013-08-12 14:45:20,714] ERROR Producer connection to kafka002:9092
>>>unsuccessful (kafka.producer.SyncProducer)
>>> java.nio.channels.ClosedByInterruptException
>>> at
>>>java.nio.channels.spi.AbstractInterruptibleChannel.end(AbstractInterrupti
>>>bleChannel.java:184)
>>> at sun.nio.ch.SocketChannelImpl.connect(SocketChannelImpl.java:543)
>>> at kafka.network.BlockingChannel.connect(BlockingChannel.scala:57)
>>
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB