Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> hiveserver leak zookeeper connections?


Copy link to this message
-
Re: hiveserver leak zookeeper connections?
@ Ashutosh:
I patched HIVE-3723 with hive 0.9 but it didn't work, I would try using
hive 0.10.0.
Thanks~
2013/1/6 Ashutosh Chauhan <[EMAIL PROTECTED]>

> We have found one source of leak which we think we have fixed.
> https://issues.apache.org/jira/browse/HIVE-3723 This is a part of
> hive-0.10 RC0 http://people.apache.org/~hashutosh/hive-0.10.0-rc0/ You
> can try hive from there and see if this fixes this connection leaking
> problem.
>
> Thanks,
> Ashutosh
>
>
> On Fri, Jan 4, 2013 at 10:51 PM, wzc <[EMAIL PROTECTED]> wrote:
>
>> Hi all:
>> I am using hive 0.9 and zookeeper 3.3.5, and I set
>> hive.support.concurrency=true for locking. when I run query through
>> hiveserver I keep receiving warnings like :
>>
>> 2013-01-05 12:03:56,109 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server data-dev-01/192.168.2.241:2181, unexpected error, closing socket connection and a
>> ttempting reconnect
>> java.io.IOException: Connection reset by peer
>>         at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>>         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>>         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
>>         at sun.nio.ch.IOUtil.read(IOUtil.java:186)
>>         at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66
>> 2013-01-05 12:03:56,350 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server data-dev-02/192.168.2.237:2181, unexpected error, closing socket connection and a
>> ttempting reconnect
>> java.io.IOException: Connection reset by peer
>>         at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>>         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>>         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
>>         at sun.nio.ch.IOUtil.read(IOUtil.java:186)
>>         at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66)
>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:291)
>>         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
>> 2013-01-05 12:03:56,476 WARN  zookeeper.ClientCnxn (ClientCnxn.java:run(1057)) - Session 0x0 for server data-dev-02/192.168.2.237:2181, unexpected error, closing socket connection and a
>> ttempting reconnect
>> java.io.IOException: Connection reset by peer
>>         at sun.nio.ch.FileDispatcherImpl.read0(Native Method)
>>         at sun.nio.ch.SocketDispatcher.read(SocketDispatcher.java:39)
>>         at sun.nio.ch.IOUtil.readIntoNativeBuffer(IOUtil.java:218)
>>         at sun.nio.ch.IOUtil.read(IOUtil.java:186)
>>         at sun.nio.ch.SocketChannelImpl.read(SocketChannelImpl.java:359)
>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doIO(ClientCnxnSocketNIO.java:66)
>>         at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:291)
>>         at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1035)
>>
>>
>> and the query fails. I run 'netstat -anlp | grep 2181' on the server which hiveserver is running and find many ESTABLISHED connections:
>>
>> tcp6       0      0 192.168.2.241:50033     192.168.2.241:2181      ESTABLISHED 11765/java
>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:50024     ESTABLISHED 22103/java
>> tcp6       0      0 192.168.2.241:59436     192.168.2.237:2181      ESTABLISHED 11765/java
>> tcp6       0      0 192.168.2.241:50015     192.168.2.241:2181      ESTABLISHED 11765/java
>> tcp6       0      0 192.168.2.241:59432     192.168.2.237:2181      ESTABLISHED 11765/java
>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:50014     ESTABLISHED 22103/java
>> tcp6       0      0 192.168.2.241:59444     192.168.2.237:2181      ESTABLISHED 11765/java
>> tcp6       0      0 192.168.2.241:2181      192.168.2.241:50029     ESTABLISHED 22103/java