Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> support checksums in HBase block


Copy link to this message
-
Re: support checksums in HBase block
On Sat, Sep 15, 2012 at 10:31 AM, jlei liu <[EMAIL PROTECTED]> wrote:
> I use Hbase0.94.1 and  hadoop-0.20.2-cdh3u5.
>
> The Hbase0.94.1 write checksums in HBase block, so we don't need to read
> checksums from metadata file of block file. But I find the BlockSender
> still read checksums from metadata file of block file every time,

How are you figuring this?   That sounds like a regression.

In 0.94.x the default is that hbase.regionserver.checksum.verify is
true which should have hbase doing checksum verify on read rather than
hdfs.

>  so the
> Hbase0.94.1 can't decrease the iops, that is right? If that is right,
> should I set the  hbase.regionserver.checksum.verify property to false in
> order to  avoid the Hbase0.94.1 to verify th checksum?
>

It would be interesting flipping the switch.  If things went faster,
then for sure there is a regression.
St.Ack
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB