Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce, mail # user - Optimizing Disk I/O - does HDFS do anything ?


Copy link to this message
-
Optimizing Disk I/O - does HDFS do anything ?
Jay Vyas 2012-11-13, 20:30
How does HDFS deal with optimization of file streaming?  Do data nodes have
any optimizations at the disk level for dealing with fragmented files?  I
assume not, but just curious if this is at all in the works, or if there
are java-y ways of dealing with a long running set of files in an HDFS
cluster.  MAybe, for example, data nodes could log the amount of time spent
on I/O for certain files as a way of reporting wether or not
defragmentation needed to be run on  a particular node in a cluster.

--
Jay Vyas
http://jayunit100.blogspot.com
+
Bertrand Dechoux 2012-11-13, 21:10
+
Scott Carey 2012-11-17, 07:27
+
Jay Vyas 2012-11-13, 21:40
+
Andy Isaacson 2012-11-13, 21:53