Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # user >> set reduced block size for a specific file


+
Ben Clay 2011-08-27, 16:33
+
Uma Maheswara Rao G 72686... 2011-08-27, 19:07
+
Ted Dunning 2011-08-27, 19:42
Copy link to this message
-
Re: set reduced block size for a specific file

On Aug 27, 2011, at 12:42 PM, Ted Dunning wrote:

> There is no way to do this for standard Apache Hadoop.

Sure there is.

You can build a custom conf dir and point it to that.  You *always* have that option for client settable options as a work around for lack of features/bugs.

1. Copy $HADOOP_CONF_DIR or $HADOOP_HOME/conf to a dir
2. modify the hdfs-site.xml to have your new block size
3. Run the following:

HADOOP_CONF_DIR=mycustomconf hadoop dfs  -put file dir

Convenient?  No.  Doable? Definitely.
+
Ben Clay 2011-08-27, 23:08
+
Aaron T. Myers 2011-08-27, 23:23
+
Praveen Sripati 2011-08-28, 03:57
+
Uma Maheswara Rao G 72686... 2011-08-28, 12:03
+
Ben Clay 2011-08-30, 10:54
+
Todd Lipcon 2011-08-30, 14:45
+
Ben Clay 2011-08-30, 15:07
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB