Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Optimizing Disk I/O - does HDFS do anything ?


Copy link to this message
-
Optimizing Disk I/O - does HDFS do anything ?
How does HDFS deal with optimization of file streaming?  Do data nodes have
any optimizations at the disk level for dealing with fragmented files?  I
assume not, but just curious if this is at all in the works, or if there
are java-y ways of dealing with a long running set of files in an HDFS
cluster.  MAybe, for example, data nodes could log the amount of time spent
on I/O for certain files as a way of reporting wether or not
defragmentation needed to be run on  a particular node in a cluster.

--
Jay Vyas
http://jayunit100.blogspot.com
+
Bertrand Dechoux 2012-11-13, 21:10
+
Scott Carey 2012-11-17, 07:27
+
Jay Vyas 2012-11-13, 21:40
+
Andy Isaacson 2012-11-13, 21:53
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB