Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> hiveserver leak zookeeper connections?


Copy link to this message
-
hiveserver leak zookeeper connections?
Hi all:
I am using hive 0.9 and zookeeper 3.3.5, and I set
hive.support.concurrency=true for locking. when I run query through
hiveserver I keep receiving warnings like :

2013-01-05 12:03:56,109 WARN  zookeeper.ClientCnxn
(ClientCnxn.java:run(1057)) - Session 0x0 for server
data-dev-01/192.168.2.241:2181, unexpected error, closing socket
connection and a
ttempting reconnect
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
        at sun.nio.ch.IOUtil.read(IOUtil.java:186)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66
2013-01-05 12:03:56,350 WARN  zookeeper.ClientCnxn
(ClientCnxn.java:run(1057)) - Session 0x0 for server
data-dev-02/192.168.2.237:2181, unexpected error, closing socket
connection and a
ttempting reconnect
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
        at sun.nio.ch.IOUtil.read(IOUtil.java:186)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:291)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
2013-01-05 12:03:56,476 WARN  zookeeper.ClientCnxn
(ClientCnxn.java:run(1057)) - Session 0x0 for server
data-dev-02/192.168.2.237:2181, unexpected error, closing socket
connection and a
ttempting reconnect
java.io.IOException: Connection reset by peer
        at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
        at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
        at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
        at sun.nio.ch.IOUtil.read(IOUtil.java:186)
        at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66)
        at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:291)
        at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
and the query fails. I run 'netstat -anlp | grep 2181' on the server
which hiveserver is running and find many ESTABLISHED connections:

tcp6       0      0 192.168.2.241:50033     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50024
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:59436     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:50015     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59432     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50014
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:59444     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50029
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50015
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:50031     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:50024     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:49998
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:59423     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:50021     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59442     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50033
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:59434     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59426     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59437     192.168.2.237:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:2181      192.168.2.241:50005
ESTABLISHED 22103/java
tcp6       0      0 192.168.2.241:49998     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:50029     192.168.2.241:2181
ESTABLISHED 11765/java
tcp6       0      0 192.168.2.241:59439     192.168.2.237:2181
ESTABLISHED 11765/java
when I turn off hive.support.concurrency option it's ok.

I think maybe hiveserver is leaking zookeeper connections. Can anyone
give me some hint?
Thanks.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB