Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # dev >> Mysterious 7 byte reads from .meta files


Copy link to this message
-
Re: Mysterious 7 byte reads from .meta files
Created https://issues.apache.org/jira/browse/HDFS-4960 and attached a
patch... Thanks !
On Sun, Jul 7, 2013 at 9:06 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> The header of the meta file is read for block metadata version
> validation. The checksum checks are performed after the header is read
> (and skipped if the config's so). The code can be viewed in the
> BlockReaderLocal.java, which is used when local reads are turned on.
>
> I suppose we could skip reading the version as well, though it seems
> "general". Please file a JIRA for this!
>
> On Sat, Jul 6, 2013 at 5:55 AM, Varun Sharma <[EMAIL PROTECTED]> wrote:
> > Hi,
> >
> > We are running hbase with application level checksums turned on. We
> > basically have the following configuration:
> >
> > dfs.client.read.shortcircuit.skip.checksum -> true
> > hbase.regionserver.checksum.verify -> true
> >
> > We have shortcircuit reads enabled and have verified that its working -
> > only the HBase region server is accessing the disk. However, an strace on
> > the region server shows some mysterious 7 byte seeks on the .meta files
> > (which are also supposed to contain the checksum). We are running
> > hadoop-2.0.0-alpha.
> >
> > Do 7 byte reads from .meta files ring a bell to someone's ears - I am not
> > totally sure if these are checksums or not and I also don't know enough
> > about hdfs ?
> >
> > Thanks
> > Varun
>
>
>
> --
> Harsh J
>