-Re: Mysterious 7 byte reads from .meta files
Varun Sharma 2013-07-07, 16:37
Created https://issues.apache.org/jira/browse/HDFS-4960 and attached a
patch... Thanks !
On Sun, Jul 7, 2013 at 9:06 AM, Harsh J <[EMAIL PROTECTED]> wrote:
> The header of the meta file is read for block metadata version
> validation. The checksum checks are performed after the header is read
> (and skipped if the config's so). The code can be viewed in the
> BlockReaderLocal.java, which is used when local reads are turned on.
> I suppose we could skip reading the version as well, though it seems
> "general". Please file a JIRA for this!
> On Sat, Jul 6, 2013 at 5:55 AM, Varun Sharma <[EMAIL PROTECTED]> wrote:
> > Hi,
> > We are running hbase with application level checksums turned on. We
> > basically have the following configuration:
> > dfs.client.read.shortcircuit.skip.checksum -> true
> > hbase.regionserver.checksum.verify -> true
> > We have shortcircuit reads enabled and have verified that its working -
> > only the HBase region server is accessing the disk. However, an strace on
> > the region server shows some mysterious 7 byte seeks on the .meta files
> > (which are also supposed to contain the checksum). We are running
> > hadoop-2.0.0-alpha.
> > Do 7 byte reads from .meta files ring a bell to someone's ears - I am not
> > totally sure if these are checksums or not and I also don't know enough
> > about hdfs ?
> > Thanks
> > Varun
> Harsh J