Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - Connection reference counting


Copy link to this message
-
Connection reference counting
Lars George 2013-07-17, 08:03
Hi,

I am working on an issue around Thrift 2 (HBASE-7035), and I am testing how this all works a bit. See https://github.com/larsgeorge/hbase-scanner-test for the code, which runs against a local HBase instance. Now, there is an issue that connections do not get closed as I would have expected. I close the connection in a loop over a scanner that gets single rows. Although I close it, it keeps on going. Scanners may or may not be reference counted - any code calling HCM.getConnection(conf) increases the count.  

So I tried a HCM.deleteAllConnections, which does this:

  public static void deleteAllConnections() {
    synchronized (HBASE_INSTANCES) {
      Set<HConnectionKey> connectionKeys = new HashSet<HConnectionKey>();
      connectionKeys.addAll(HBASE_INSTANCES.keySet());
      for (HConnectionKey connectionKey : connectionKeys) {
        deleteConnection(connectionKey, false);
      }
      HBASE_INSTANCES.clear();
    }
  }

It iterates over all connections and "deletes" them. Then is clears the entire reference list! The issue is that deleteConnection() is using the refcounts and is *not* closing the connection when it is still referenced. The final clear() call simply drops them from the list of managed connections. This means that we now might have dangling open connections, and unless you hold a reference there is no way that you can talk to it again, not even using deleteStaleConnection() for example.

Is that wanted? Just curious, since obviously this is not a biggie in real life.

Lars

+
Nick Dimiduk 2013-07-17, 16:35
+
Nicolas Liochon 2013-07-17, 16:31
+
Lars George 2013-07-17, 17:06
+
Nicolas Liochon 2013-07-17, 17:22