Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase CheckSum vs Hadoop CheckSum


Copy link to this message
-
答复: HBase CheckSum vs Hadoop CheckSum
comments in line

Regards,
Liang
________________________________________
发件人: Jean-Marc Spaggiari [[EMAIL PROTECTED]]
发送时间: 2013年2月26日 20:03
收件人: user
主题: HBase CheckSum vs Hadoop CheckSum

Hi,

Quick question.

When we are activating the short circuit read in HBase, it's
recommanded to activate the HBase checksum instead of Hadoop ones.
This is done in the HBase configuration.

I'm wondering what is the impact on the DataNode Block Scanner.

Is it going to be stopped because checksums can't be used anymore? Or
will Hadoop continue to store its own checksum and use them but it's
just that HBase will not look at them anymore and will store and use
its own checksums?
[liang xie]: yes, still store checksum in meta file in current community version.
btw, facebook's hadoop-fb20 branch has an inline checksum feature,IIRC

Since it's an HBase configuration (hbase.regionserver.checksum.verify)
I'm expecting this to not have any impact on the Block Scanner, but
I'm looking for a confirmation.
[liang xie]: yes, no impact on hdfs's DataBlockScanner, you can check
detail in datanode's BlockPoolSliceScanner.verifyBlock():
        blockSender = new BlockSender(block, 0, -1, false, true, true,
            datanode, null);
Thanks,

JM
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB