Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Mysterious 7 byte reads from .meta files


Copy link to this message
-
Re: Mysterious 7 byte reads from .meta files
The header of the meta file is read for block metadata version
validation. The checksum checks are performed after the header is read
(and skipped if the config's so). The code can be viewed in the
BlockReaderLocal.java, which is used when local reads are turned on.

I suppose we could skip reading the version as well, though it seems
"general". Please file a JIRA for this!

On Sat, Jul 6, 2013 at 5:55 AM, Varun Sharma <[EMAIL PROTECTED]> wrote:
> Hi,
>
> We are running hbase with application level checksums turned on. We
> basically have the following configuration:
>
> dfs.client.read.shortcircuit.skip.checksum -> true
> hbase.regionserver.checksum.verify -> true
>
> We have shortcircuit reads enabled and have verified that its working -
> only the HBase region server is accessing the disk. However, an strace on
> the region server shows some mysterious 7 byte seeks on the .meta files
> (which are also supposed to contain the checksum). We are running
> hadoop-2.0.0-alpha.
>
> Do 7 byte reads from .meta files ring a bell to someone's ears - I am not
> totally sure if these are checksums or not and I also don't know enough
> about hdfs ?
>
> Thanks
> Varun

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB