Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Question about dfs.block.size setting


Copy link to this message
-
Question about dfs.block.size setting
Hi all,

There're lots of materials from internet suggest to set dfs.block.size
larger, e.g. from 64M to 256M, when the job is large. And they said the
performance would improve. But I'm not clear why increse the block size will
improve. I know that increase block size will reduce the map task number for
the same input, but why lesser map tasks will improve overall performance?

Any comments would be highly valued, and thanks in advance.

Best Regards,
Carp
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB