Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Producer can't initialize if zk host down (0.7.2)


Copy link to this message
-
Re: Producer can't initialize if zk host down (0.7.2)
Neha Narkhede 2013-05-21, 21:20
According to zookeeper code, when you try to create a client handle to
zookeeper, it resolves each of the hosts in the zookeeper connection
string. If any of these fail, it throws an exception. If you use a
zookeeper based producer, it tries to create a ZkClient handle which in
turn tries to create a ZooKeeper handle. Since the ZooKeeper handle has to
resolve every host on startup, you have to make sure that is possible.

Thanks,
Neha
On Tue, May 21, 2013 at 11:09 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am seeing an unexpected situation.  My producers use a zkconnection
> string to connect to kafka (this is still 0.7.2).  If one of the zk hosts
> is taken down and removed from dns, it causes an UnknownHostException, and
> the producer can't initialize.  I expect this is different than the less
> severe case where one of the zk's is not responding, but the host still
> exists in dns, etc.  We are in the process or moving some of our zk
> cluster, but I had hoped that it would be ok to do in stages....e.g. as
> long as most of the zk's are available in a client's connection string, it
> should be ok for it proceed, without having to proactively update and
> restart each client with the new zk connect string.
>
> However, this appears not to be the case (see exception below).  Is this
> expected?
>
> org.I0Itec.zkclient.exception.ZkException: Unable to connect to <redacted>
>         at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:66)
>         at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
>         at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>         at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>         at
> kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:62)
>         at kafka.producer.Producer.<init>(Producer.scala:47)
>         at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
>         at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
>         ......
> Caused by: java.net.UnknownHostException: <redacted>
>         at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
>         at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:850)
>         at
> java.net.InetAddress.getAddressFromNameService(InetAddress.java:1201)
>         at java.net.InetAddress.getAllByName0(InetAddress.java:1154)
>         at java.net.InetAddress.getAllByName(InetAddress.java:1084)
>         at java.net.InetAddress.getAllByName(InetAddress.java:1020)
>         at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:382)
>         at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:327)
>         at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
>         at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
>         ... 19 more
>
> Jason
>