Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Error about rs block seek


Copy link to this message
-
Re: Error about rs block seek
ramkrishna vasudevan 2013-05-13, 08:34
Is it possible to reproduce this with simple test case based on your
usecase and data? You can share it so that can really debug the actual
problem.
Regards
Ram
On Mon, May 13, 2013 at 1:57 PM, Anoop John <[EMAIL PROTECTED]> wrote:

> >So I want to know if I set block size at the beginning of creating tables,
> does something make troubles?
>
> Should not. We have tested with diff block sizes from def 64K to 8K fro
> testing purposes.  Have not came across issues like this.  Only on this
> data it is coming or every time u create a new table with 32K as block size
> and do some writes and then do read, this issue comes?
>
> -Anoop-
>
> On Mon, May 13, 2013 at 1:36 PM, Bing Jiang <[EMAIL PROTECTED]
> >wrote:
>
> > hi, Anoop.
> > I do not handle or change the hbase checksum.
> >
> > So I want to know if I set block size at the beginning of creating
> tables,
> > does something make troubles?
> >
> >
> > 2013/5/13 Anoop John <[EMAIL PROTECTED]>
> >
> > > > Current pos = 32651;
> > > currKeyLen = 45; currValLen = 80; block limit = 32775
> > >
> > > This means after the cur position we need to have atleast  45+80+4(key
> > > length stored as 4 bytes) +4(value length 4 bytes)
> > > So atleast 32784 should have been the limit.  If we have memstoreTS
> also
> > > written with this KV some more bytes..
> > >
> > > Do u use Hbase handled checksum?
> > >
> > > -Anoop-
> > >
> > > On Mon, May 13, 2013 at 12:00 PM, Bing Jiang <[EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Hi,all
> > > > Before the exception stack, there is an Error log:
> > > > 2013-05-13 00:00:14,491 ERROR
> > > > org.apache.hadoop.hbase.io.hfile.HFileReaderV2: Current pos = 32651;
> > > > currKeyLen = 45; currValLen = 80; block limit = 32775; HFile name > > > > 1f96183d55144c058fa2a05fe5c0b814; currBlock currBlockOffset > 33550830
> > > >
> > > > And the operation is scanner's next.
> > > > Current pos + currKeyLen + currValLen > block limit
> > > > 32651+45 +80 = 32776 > 32775 , and in my table configs, set blocksize
> > > > 32768, and when I change the value from blocksize from 64k(default
> > value)
> > > > to 32k, so many error logs being found.
> > > >
> > > > I use 0.94.3, can someone tell me the influence of blocksize setting.
> > > >
> > > > Tks.
> > > >
> > > >
> > > >
> > > >
> > > > 2013/5/13 ramkrishna vasudevan <[EMAIL PROTECTED]>
> > > >
> > > > > Your TTL is negative here 'TTL => '-1','.
> > > > >
> > > > > Any reason for it to be negative? This could be a possible reason.
> >  Not
> > > > > sure..
> > > > >
> > > > > Regards
> > > > > Ram
> > > > >
> > > > >
> > > > > On Mon, May 13, 2013 at 7:20 AM, Bing Jiang <
> > [EMAIL PROTECTED]
> > > > > >wrote:
> > > > >
> > > > > > hi, Ted.
> > > > > >
> > > > > > No data block encoding, our table config below:
> > > > > >
> > > > > > User Table Description
> > > > > > CrawlInfo<http://10.100.12.33:8003/table.jsp?name=CrawlInfo>
> {NAME
> > > > > > => 'CrawlInfo', DEFERRED_LOG_FLUSH => 'true', MAX_FILESIZE =>
> > > > > > '34359738368', FAMILIES => [{NAME => 'CrawlStats', BLOOMFILTER =>
> > > > > 'ROWCOL',
> > > > > > CACHE_INDEX_ON_WRITE => 'true', TTL => '-1', CACHE_DATA_ON_WRITE
> =>
> > > > > 'true',
> > > > > > CACHE_BLOOMS_ON_WRITE => 'true', VERSIONS => '1', BLOCKSIZE =>
> > > > '32768'}]}
> > > > > >
> > > > > >
> > > > > >
> > > > > > 2013/5/13 Bing Jiang <[EMAIL PROTECTED]>
> > > > > >
> > > > > > > Hi, JM.
> > > > > > > Our jdk version is 1.6.0_38
> > > > > > >
> > > > > > >
> > > > > > > 2013/5/13 Jean-Marc Spaggiari <[EMAIL PROTECTED]>
> > > > > > >
> > > > > > >> Hi Bing,
> > > > > > >>
> > > > > > >> Which JDK are you using?
> > > > > > >>
> > > > > > >> Thanks,
> > > > > > >>
> > > > > > >> JM
> > > > > > >>
> > > > > > >> 2013/5/12 Bing Jiang <[EMAIL PROTECTED]>
> > > > > > >>
> > > > > > >> > Yes, we use hbase-0.94.3 , and  we change block.size from
> 64k
> > to
> > > > > 32k.
> > > > > > >> >
> > > > > > >> >
> > > > > > >> > 2013/5/13 Ted Yu <[EMAIL PROTECTED]>