Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS, mail # user - set reduced block size for a specific file


+
Ben Clay 2011-08-27, 16:33
+
Uma Maheswara Rao G 72686... 2011-08-27, 19:07
+
Ted Dunning 2011-08-27, 19:42
Copy link to this message
-
Re: set reduced block size for a specific file
Allen Wittenauer 2011-08-27, 22:41

On Aug 27, 2011, at 12:42 PM, Ted Dunning wrote:

> There is no way to do this for standard Apache Hadoop.

Sure there is.

You can build a custom conf dir and point it to that.  You *always* have that option for client settable options as a work around for lack of features/bugs.

1. Copy $HADOOP_CONF_DIR or $HADOOP_HOME/conf to a dir
2. modify the hdfs-site.xml to have your new block size
3. Run the following:

HADOOP_CONF_DIR=mycustomconf hadoop dfs  -put file dir

Convenient?  No.  Doable? Definitely.
+
Ben Clay 2011-08-27, 23:08
+
Aaron T. Myers 2011-08-27, 23:23
+
Praveen Sripati 2011-08-28, 03:57
+
Uma Maheswara Rao G 72686... 2011-08-28, 12:03
+
Ben Clay 2011-08-30, 10:54
+
Todd Lipcon 2011-08-30, 14:45
+
Ben Clay 2011-08-30, 15:07