Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Producer can't initialize if zk host down (0.7.2)


Copy link to this message
-
Re: Producer can't initialize if zk host down (0.7.2)
According to zookeeper code, when you try to create a client handle to
zookeeper, it resolves each of the hosts in the zookeeper connection
string. If any of these fail, it throws an exception. If you use a
zookeeper based producer, it tries to create a ZkClient handle which in
turn tries to create a ZooKeeper handle. Since the ZooKeeper handle has to
resolve every host on startup, you have to make sure that is possible.

Thanks,
Neha
On Tue, May 21, 2013 at 11:09 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am seeing an unexpected situation.  My producers use a zkconnection
> string to connect to kafka (this is still 0.7.2).  If one of the zk hosts
> is taken down and removed from dns, it causes an UnknownHostException, and
> the producer can't initialize.  I expect this is different than the less
> severe case where one of the zk's is not responding, but the host still
> exists in dns, etc.  We are in the process or moving some of our zk
> cluster, but I had hoped that it would be ok to do in stages....e.g. as
> long as most of the zk's are available in a client's connection string, it
> should be ok for it proceed, without having to proactively update and
> restart each client with the new zk connect string.
>
> However, this appears not to be the case (see exception below).  Is this
> expected?
>
> org.I0Itec.zkclient.exception.ZkException: Unable to connect to <redacted>
>         at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:66)
>         at org.I0Itec.zkclient.ZkClient.connect(ZkClient.java:872)
>         at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:98)
>         at org.I0Itec.zkclient.ZkClient.<init>(ZkClient.java:84)
>         at
> kafka.producer.ZKBrokerPartitionInfo.<init>(ZKBrokerPartitionInfo.scala:62)
>         at kafka.producer.Producer.<init>(Producer.scala:47)
>         at kafka.javaapi.producer.Producer.<init>(Producer.scala:33)
>         at kafka.javaapi.producer.Producer.<init>(Producer.scala:40)
>         ......
> Caused by: java.net.UnknownHostException: <redacted>
>         at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
>         at java.net.InetAddress$1.lookupAllHostAddr(InetAddress.java:850)
>         at
> java.net.InetAddress.getAddressFromNameService(InetAddress.java:1201)
>         at java.net.InetAddress.getAllByName0(InetAddress.java:1154)
>         at java.net.InetAddress.getAllByName(InetAddress.java:1084)
>         at java.net.InetAddress.getAllByName(InetAddress.java:1020)
>         at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:382)
>         at org.apache.zookeeper.ClientCnxn.<init>(ClientCnxn.java:327)
>         at org.apache.zookeeper.ZooKeeper.<init>(ZooKeeper.java:383)
>         at org.I0Itec.zkclient.ZkConnection.connect(ZkConnection.java:64)
>         ... 19 more
>
> Jason
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB