Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> optimizing HDFS writes with replication=1


Copy link to this message
-
optimizing HDFS writes with replication=1
I am writing temp files to HDFS with replication=1, so I expect the blocks to be stored on the writing node.  Are there any tips, in general, for optimizing write performance to HDFS?  I use 128K buffers in the write() calls.  Are there any parameters that can be set  on the connection or in HDFS configuration to optimize this use pattern?
Thanks
John