Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # dev - HDFS pread performance


Copy link to this message
-
HDFS pread performance
lei liu 2013-04-17, 13:54
I test the HDFS pread performance, the avg time of pread is about 10ms,
but  pread max time reach 200ms, there is about one percent of pread
time is 200ms, that result to my application is timeout.

  I find  max time of the
RemoteBlockReader<eclipse-javadoc:%E2%98%82=hadoop-0.20.2-cdh3u5/src%5C/hdfs%3Corg.apache.hadoop.hdfs%7BDFSClient.java%E2%98%83DFSClient%E2%98%83RemoteBlockReader>.readChunk
method also can reach 100ms.  The
RemoteBlockReader<eclipse-javadoc:%E2%98%82=hadoop-0.20.2-cdh3u5/src%5C/hdfs%3Corg.apache.hadoop.hdfs%7BDFSClient.java%E2%98%83DFSClient%E2%98%83RemoteBlockReader>.readChunk
only read 512 bytes form socket, so  I think I can tuning linux socket
Kernel parameters to reduce the max time of pread. If anybody has the
experience,
please tell me.
Thansk,

LiuLei