Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Producer can't initialize if zk host down (0.7.2)


Copy link to this message
-
Producer can't initialize if zk host down (0.7.2)
Hi,

I am seeing an unexpected situation.  My producers use a zkconnection
string to connect to kafka (this is still 0.7.2).  If one of the zk hosts
is taken down and removed from dns, it causes an UnknownHostException, and
the producer can't initialize.  I expect this is different than the less
severe case where one of the zk's is not responding, but the host still
exists in dns, etc.  We are in the process or moving some of our zk
cluster, but I had hoped that it would be ok to do in stages....e.g. as
long as most of the zk's are available in a client's connection string, it
should be ok for it proceed, without having to proactively update and
restart each client with the new zk connect string.

However, this appears not to be the case (see exception below).  Is this
expected?

org.I0Itec.zkclient.exception.ZkException: Unable to connect to <redacted>
        at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:66)
        at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
        at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
        at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
        at
kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:62)
        at kafka.producer.Producer.<init>(Producer.scala:47)
        at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
        at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
        ......
Caused by: java.net.UnknownHostException: <redacted>
        at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
        at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:850)
        at
java.net.InetAddress.getAddressFromNameService(InetAddress.java:1201)
        at java.net.InetAddress.getAllByName0(InetAddress.java:1154)
        at java.net.InetAddress.getAllByName(InetAddress.java:1084)
        at java.net.InetAddress.getAllByName(InetAddress.java:1020)
        at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:382)
        at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:327)
        at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
        at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
        ... 19 more

Jason

 
+
Neha Narkhede 2013-05-21, 21:20
+
Jason Rosenberg 2013-05-21, 22:05
+
Neha Narkhede 2013-05-21, 22:38