Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> hiveserver leak zookeeper connections?


Copy link to this message
-
hiveserver leak zookeeper connections?
Hi all:
I am using hive 0.9 and zookeeper 3.3.5, and I set
hive.support.concurrency=true for locking. when I run query through
hiveserver I keep receiving warnings like :

2013-01-05 12:03:56,109 WARN  zookeeper.ClientCnxn
(ClientCnxn.java:run(1057)) - Session 0x0 for server
data-dev-01/192.168.2.241:2181, unexpected error, closing socket
connection and a
ttempting reconnect
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
        at sun.nio.ch.IOUtil.read(IOUtil.java:186)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66
2013-01-05 12:03:56,350 WARN  zookeeper.ClientCnxn
(ClientCnxn.java:run(1057)) - Session 0x0 for server
data-dev-02/192.168.2.237:2181, unexpected error, closing socket
connection and a
ttempting reconnect
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
        at sun.nio.ch.IOUtil.read(IOUtil.java:186)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:291)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2013-01-05 12:03:56,476 WARN  zookeeper.ClientCnxn
(ClientCnxn.java:run(1057)) - Session 0x0 for server
data-dev-02/192.168.2.237:2181, unexpected error, closing socket
connection and a
ttempting reconnect
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
        at sun.nio.ch.IOUtil.read(IOUtil.java:186)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:291)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
and the query fails. I run 'netstat -anlp | grep 2181' on the server
which hiveserver is running and find many ESTABLISHED connections:

tcp6       0      0 192.168.2.241:50033     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50024
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:59436     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:50015     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59432     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50014
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:59444     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50029
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50015
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:50031     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:50024     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:49998
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:59423     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:50021     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59442     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50033
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:59434     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59426     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59437     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50005
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:49998     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:50029     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59439     192.168.2.237:2181
ESTABLISHED 11765/java
when I turn off hive.support.concurrency option it's ok.

I think maybe hiveserver is leaking zookeeper connections. Can anyone
give me some hint?
Thanks.