I am seeing an unexpected situation. My producers use a zkconnection string to connect to kafka (this is still 0.7.2). If one of the zk hosts is taken down and removed from dns, it causes an UnknownHostException, and the producer can't initialize. I expect this is different than the less severe case where one of the zk's is not responding, but the host still exists in dns, etc. We are in the process or moving some of our zk cluster, but I had hoped that it would be ok to do in stages....e.g. as long as most of the zk's are available in a client's connection string, it should be ok for it proceed, without having to proactively update and restart each client with the new zk connect string.
However, this appears not to be the case (see exception below). Is this expected?
org.I0Itec.zkclient.exception.ZkException: Unable to connect to <redacted> at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:66) at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872) at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98) at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84) at kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:62) at kafka.producer.Producer.<init>(Producer.scala:47) at kafka.javaapi.producer.Producer.<init>(Producer.scala:33) at kafka.javaapi.producer.Producer.<init>(Producer.scala:40) ...... Caused by: java.net.UnknownHostException: <redacted> at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method) at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:850) at java.net.InetAddress.getAddressFromNameService(InetAddress.java:1201) at java.net.InetAddress.getAllByName0(InetAddress.java:1154) at java.net.InetAddress.getAllByName(InetAddress.java:1084) at java.net.InetAddress.getAllByName(InetAddress.java:1020) at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:382) at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:327) at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383) at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64) ... 19 more
According to zookeeper code, when you try to create a client handle to zookeeper, it resolves each of the hosts in the zookeeper connection string. If any of these fail, it throws an exception. If you use a zookeeper based producer, it tries to create a ZkClient handle which in turn tries to create a ZooKeeper handle. Since the ZooKeeper handle has to resolve every host on startup, you have to make sure that is possible.
Thanks, Neha On Tue, May 21, 2013 at 11:09 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
I see now that is true. It looks like it just needs to resolve all the hosts (but doesn't need zk to be available on each host). So decommissioning a zknode (and removing from dns) is a bit more sensitive than I thought. I can't think of any reason that zkClient should require this behavior, other than for keeping clean configurations. But it makes it difficult to roll in new changes to the cluster layout while keeping services up.
Jason On Tue, May 21, 2013 at 2:20 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
Right. Zookeeper has added the feature to allow dynamically reconfiguring zookeeper clusters, but it is part of zookeeper 3.5.0.
Thanks, Neha On Tue, May 21, 2013 at 3:05 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext