Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HDFS >> mail # dev >> Problem reading HDFS block size > 1GB


+
Vinay Setty 2009-09-29, 17:59
Copy link to this message
-
Re: Problem reading HDFS block size > 1GB

On Sep 29, 2009, at 10:59 AM, Vinay Setty wrote:

> We are running Yahoo distribution of Hadoop based on Hadoop  
> 0.20.0-2787265 .
> On a 10 nodes cluster with OpenSUSE Linux Operating System. We have  
> HDFS
> configured with Block Size 5GB (This is for our experiments).

There is a known limitation to HDFS to blocks of less than 2^31 bytes.  
Fixing it would be tedious and no one has signed up to take a pass at  
it.

-- Owen
+
Vinay Setty 2009-09-29, 21:14
+
Brian Bockelman 2009-09-29, 21:58
+
Dhruba Borthakur 2009-09-29, 22:16
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB