Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Connection reference counting


Copy link to this message
-
Re: Connection reference counting
I had a look, I think it's a bug.
If the connection holds resources that should be freed, well they won't be
freed.
It's used in the minicluster shutdown. So we could imagine scenarios which
that create some test flakyness. I doubt it happens in real life, but imho
it's better to fix it (it there are no different opinions, I will do it if
you don't ;-).

Cheers,

Nicolas
On Wed, Jul 17, 2013 at 10:03 AM, Lars George <[EMAIL PROTECTED]> wrote:

> Hi,
>
> I am working on an issue around Thrift 2 (HBASE-7035), and I am testing
> how this all works a bit. See
> https://github.com/larsgeorge/hbase-scanner-test for the code, which runs
> against a local HBase instance. Now, there is an issue that connections do
> not get closed as I would have expected. I close the connection in a loop
> over a scanner that gets single rows. Although I close it, it keeps on
> going. Scanners may or may not be reference counted - any code calling
> HCM.getConnection(conf) increases the count.
>
> So I tried a HCM.deleteAllConnections, which does this:
>
>   public static void deleteAllConnections() {
>     synchronized (HBASE_INSTANCES) {
>       Set<HConnectionKey> connectionKeys = new HashSet<HConnectionKey>();
>       connectionKeys.addAll(HBASE_INSTANCES.keySet());
>       for (HConnectionKey connectionKey : connectionKeys) {
>         deleteConnection(connectionKey, false);
>       }
>       HBASE_INSTANCES.clear();
>     }
>   }
>
> It iterates over all connections and "deletes" them. Then is clears the
> entire reference list! The issue is that deleteConnection() is using the
> refcounts and is *not* closing the connection when it is still referenced.
> The final clear() call simply drops them from the list of managed
> connections. This means that we now might have dangling open connections,
> and unless you hold a reference there is no way that you can talk to it
> again, not even using deleteStaleConnection() for example.
>
> Is that wanted? Just curious, since obviously this is not a biggie in real
> life.
>
> Lars
>
>
>
>
>
>