-Too Many CLOSE_WAIT make performance down
Bing Jiang 2013-08-07, 06:03
There are too many CLOSE_WAIT connection from RS to DN, and I find the
number is over 30000.
And change the Log-Level of 'org.apache.hadoop.ipc.HBaseServer.trace' to
DEBUG, and check that the performance:
> Call #2649932; Served: HRegionInterface#get queueTime=0 processingTime=284
> contents=1 Get, 86 bytes
> So the conclusion is that when DataNode server port has been occupied by
normal or irregular connection, it will bring read/write performance down.
According to principle of TCP/IP protocol, CLOSE_WAIT means that RS cannot
close fd which has been opened, and I restart RS gracefully, the problem
has been tackled. Ok, My question is :
Can someone tell me in which conditions do RS will ignore the file handler?
Any ideas will be nice.
Focus on distributed computing, HDFS/HBase