Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # dev >> Review Request 14978: HIVE-5643: ZooKeeperHiveLockManager.getQuorumServers incorrectly appends the custom zk port to quorum hosts


Copy link to this message
-
Review Request 14978: HIVE-5643: ZooKeeperHiveLockManager.getQuorumServers incorrectly appends the custom zk port to quorum hosts

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14978/
-----------------------------------------------------------

Review request for hive and Brock Noland.
Bugs: HIVE-5643
    https://issues.apache.org/jira/browse/HIVE-5643
Repository: hive-git
Description
-------

ZooKeeperHiveLockManager calls the below method to construct the connection string for ZooKeeper connection.

  private static String getQuorumServers(HiveConf conf) {
    String hosts = conf.getVar(HiveConf.ConfVars.HIVE_ZOOKEEPER_QUORUM);
    String port = conf.getVar(HiveConf.ConfVars.HIVE_ZOOKEEPER_CLIENT_PORT);
    return hosts + ":" + port;
  }

For example:
HIVE_ZOOKEEPER_QUORUM=node1, node2, node3
HIVE_ZOOKEEPER_CLIENT_PORT=9999

Connection string given to ZooKeeper client object is "node1, node2, node3:9999". ZooKeeper consider the default port as 2181 for hostnames that don't have any port. This works fine as long as HIVE_ZOOKEEPER_CLIENT_PORT is 2181. If it is different then ZooKeeper client object tries to connect to node1 and node2 on port 2181 which always fails. So it has only one choice the last host which receives all the load from Hive.
Diffs
-----

  ql/src/java/org/apache/hadoop/hive/ql/lockmgr/zookeeper/ZooKeeperHiveLockManager.java 735e745
  ql/src/test/org/apache/hadoop/hive/ql/lockmgr/zookeeper/TestZookeeperLockManager.java 2ff48f5

Diff: https://reviews.apache.org/r/14978/diff/
Testing
-------

Added a unittest for getQuorumServers which test different types of quorum settings.
Thanks,

Venki Korukanti