I use Hbase0.94.1 and  hadoop-0.20.2-cdh3u5.

The Hbase0.94.1 write checksums in HBase block, so we don't need to read
checksums from metadata file of block file. But I find the BlockSender
still read checksums from metadata file of block file every time,  so the
Hbase0.94.1 can't decrease the iops, that is right? If that is right,
should I set the  hbase.regionserver.checksum.verify property to false in
order to  avoid the Hbase0.94.1 to verify th checksum?
Thanks,
LiuLei

On Sat, Sep 15, 2012 at 10:31 AM, jlei liu <[EMAIL PROTECTED]> wrote:
> I use Hbase0.94.1 and  hadoop-0.20.2-cdh3u5.
>
> The Hbase0.94.1 write checksums in HBase block, so we don't need to read
> checksums from metadata file of block file. But I find the BlockSender
> still read checksums from metadata file of block file every time,

How are you figuring this?   That sounds like a regression.

In 0.94.x the default is that hbase.regionserver.checksum.verify is
true which should have hbase doing checksum verify on read rather than
hdfs.

>  so the
> Hbase0.94.1 can't decrease the iops, that is right? If that is right,
> should I set the  hbase.regionserver.checksum.verify property to false in
> order to  avoid the Hbase0.94.1 to verify th checksum?
>

It would be interesting flipping the switch.  If things went faster,
then for sure there is a regression.
St.Ack

The HBase support checksums in HBase block cache in
HBASE-5074<https://issues.apache.org/jira/browse/HBASE-5074>
jira.
The  HBase  support checksums for decrease the iops of  HDFS, so that HDFS
dont't need to read the metadata of block file. But in hadoop-0.20.2-cdh3u5
version, BlockSender still read the metadata file even if the
hbase.regionserver.checksum.verify property is ture. Because the HBASE-5074
jira can't decrease the iops of HDFS, so I want to set
hbase.regionserver.checksum.verify property to fasle.
2012/9/16 Stack <[EMAIL PROTECTED]>

> On Sat, Sep 15, 2012 at 10:31 AM, jlei liu <[EMAIL PROTECTED]> wrote:
> > I use Hbase0.94.1 and  hadoop-0.20.2-cdh3u5.
> >
> > The Hbase0.94.1 write checksums in HBase block, so we don't need to read
> > checksums from metadata file of block file. But I find the BlockSender
> > still read checksums from metadata file of block file every time,
>
> How are you figuring this?   That sounds like a regression.
>
> In 0.94.x the default is that hbase.regionserver.checksum.verify is
> true which should have hbase doing checksum verify on read rather than
> hdfs.
>
> >  so the
> > Hbase0.94.1 can't decrease the iops, that is right? If that is right,
> > should I set the  hbase.regionserver.checksum.verify property to false in
> > order to  avoid the Hbase0.94.1 to verify th checksum?
> >
>
> It would be interesting flipping the switch.  If things went faster,
> then for sure there is a regression.
> St.Ack
>

On Sat, Sep 15, 2012 at 7:06 PM, jlei liu <[EMAIL PROTECTED]> wrote:
> The HBase support checksums in HBase block cache in
> HBASE-5074<https://issues.apache.org/jira/browse/HBASE-5074>
> jira.
> The  HBase  support checksums for decrease the iops of  HDFS, so that HDFS
> dont't need to read the metadata of block file. But in hadoop-0.20.2-cdh3u5
> version, BlockSender still read the metadata file even if the
> hbase.regionserver.checksum.verify property is ture. Because the HBASE-5074
> jira can't decrease the iops of HDFS, so I want to set
> hbase.regionserver.checksum.verify property to fasle.
> 2012/9/16 Stack <[EMAIL PROTECTED]>
>

Go ahead, but, it seems like something is broke?  We should file an
issue to fix this broken checksuming?
St.Ack