Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: 0.8 throwing exception "Failed to find leader" and high-level consumer fails to make progress

Copy link to this message
Re: 0.8 throwing exception "Failed to find leader" and high-level consumer fails to make progress
This typically only happens when the consumerConnector is shut down. Are
you restarting the consumerConnector often?


On Tue, Jun 25, 2013 at 9:40 AM, Hargett, Phil <

> Seeing this exception a LOT (3-4 times per second, same log topic).
> I'm using external code to feed data to about 50 different log topics over
> a cluster of 3 Kafka 0.8 brokers.  There are 3 ZooKeeper instances as well,
> all of this is running on EC2.  My application creates a high-level
> consumer (1 per topic) to consumer data from each and do further processing.
> The problem is this exception is in the high-level consumer, so my code
> has no way of knowing that it's become stuck.
> This exception does not always appear, but as far as I can tell, once this
> happens, the only cure is to restart my application's process.
> I saw this in 0.8 built from source about 1 week ago, and also am seeing
> it today after pulling the latest 0.8 sources and rebuilding Kafka.
> Thoughts?
> Failed to find leader for Set([topic6,0]): java.lang.NullPointerException
>         at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:416)
>         at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:413)
>         at
> org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
>         at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:413)
>         at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:409)
>         at
> kafka.utils.ZkUtils$.getChildrenParentMayNotExist(ZkUtils.scala:438)
>         at kafka.utils.ZkUtils$.getAllBrokersInCluster(ZkUtils.scala:75)
>         at
> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:63)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)