Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Problem reading HDFS block size > 1GB

Copy link to this message
Re: Problem reading HDFS block size > 1GB

On Sep 29, 2009, at 10:59 AM, Vinay Setty wrote:

> We are running Yahoo distribution of Hadoop based on Hadoop  
> 0.20.0-2787265 .
> On a 10 nodes cluster with OpenSUSE Linux Operating System. We have  
> configured with Block Size 5GB (This is for our experiments).

There is a known limitation to HDFS to blocks of less than 2^31 bytes.  
Fixing it would be tedious and no one has signed up to take a pass at  

-- Owen