Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS, mail # user - changing the block size


Copy link to this message
-
Re: changing the block size
Allen Wittenauer 2011-02-08, 00:16

On Feb 6, 2011, at 2:24 PM, Rita wrote:
> So, what I did was decommission a node, remove all of its data (rm -rf
> data.dir) and stopped the hdfs process on it. Then I made the change to
> conf/hdfs-site.xml on the data node and then I restarted the datanode. I
> then ran a balancer to take effect and I am still getting 64MB files instead
> of 128MB. :-/
Right.

As previously mentioned, changing the block size does not change the blocks of the previously written files.  In other words, changing the block size does not act as a merging function at the datanode level.  In order to change pre-existing files, you'll need to copy the files to a new location, delete the old ones, then mv the new versions back.