Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: 0.8 throwing exception "Failed to find leader" and high-level consumer fails to make progress


Copy link to this message
-
Re: 0.8 throwing exception "Failed to find leader" and high-level consumer fails to make progress
This typically only happens when the consumerConnector is shut down. Are
you restarting the consumerConnector often?

Thanks,

Jun
On Tue, Jun 25, 2013 at 9:40 AM, Hargett, Phil <
[EMAIL PROTECTED]> wrote:

> Seeing this exception a LOT (3-4 times per second, same log topic).
>
> I'm using external code to feed data to about 50 different log topics over
> a cluster of 3 Kafka 0.8 brokers.  There are 3 ZooKeeper instances as well,
> all of this is running on EC2.  My application creates a high-level
> consumer (1 per topic) to consumer data from each and do further processing.
>
> The problem is this exception is in the high-level consumer, so my code
> has no way of knowing that it's become stuck.
>
> This exception does not always appear, but as far as I can tell, once this
> happens, the only cure is to restart my application's process.
>
> I saw this in 0.8 built from source about 1 week ago, and also am seeing
> it today after pulling the latest 0.8 sources and rebuilding Kafka.
>
> Thoughts?
>
> Failed to find leader for Set([topic6,0]): java.lang.NullPointerException
>         at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:416)
>         at org.I0Itec.zkclient.ZkClient$2.call(ZkClient.java:413)
>         at
> org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:675)
>         at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:413)
>         at org.I0Itec.zkclient.ZkClient.getChildren(ZkClient.java:409)
>         at
> kafka.utils.ZkUtils$.getChildrenParentMayNotExist(ZkUtils.scala:438)
>         at kafka.utils.ZkUtils$.getAllBrokersInCluster(ZkUtils.scala:75)
>         at
> kafka.consumer.ConsumerFetcherManager$LeaderFinderThread.doWork(ConsumerFetcherManager.scala:63)
>         at kafka.utils.ShutdownableThread.run(ShutdownableThread.scala:51)
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB