Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Why is this region compacting?


Copy link to this message
-
Re: Why is this region compacting?
TTL seems to be fine.

-1 is the default value for TimeRangeTracker.maximumTimestamp.

Can you run:
hadoop fs -lsr hdfs://
hdpmgr001.pse.movenetworks.com:8020/hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/

Thanks,

JM
2013/9/24 Tom Brown <[EMAIL PROTECTED]>

> 1. Hadoop version is 1.1.2.
> 2. All servers are synched with NTP.
> 3. Table definition is: 'compound0', {
> NAME => 'd',
> DATA_BLOCK_ENCODING => 'NONE',
> BLOOMFILTER => 'ROW',
> REPLICATION_SCOPE => '0',
> VERSIONS => '1',
> COMPRESSION => 'SNAPPY',
> MIN_VERSIONS => '0',
> TTL => '8640000',
> KEEP_DELETED_CELLS => 'false',
> BLOCKSIZE => '65536',
> IN_MEMORY => 'false',
> ENCODE_ON_DISK => 'true',
> BLOCKCACHE => 'true'
> }
>
> The TTL is supposed to be 100 days.
>
> --Tom
>
>
> On Tue, Sep 24, 2013 at 10:53 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
> > Another important information why might be the root cause of this
> issue...
> >
> > Do you have any TTL defines for this table?
> >
> > JM
> >
> >
> > 2013/9/24 Jean-Marc Spaggiari <[EMAIL PROTECTED]>
> >
> > > Strange.
> > >
> > > Few questions then.
> > > 1) What is your hadoop version?
> > > 2) Is the clock on all your severs synched with NTP?
> > > 3) What is you table definition? Bloom filters, etc.?
> > >
> > > This is the reason why it keep compacting:
> > >
> > > 2013-09-24 10:04:00,548 INFO
> > org.apache.hadoop.hbase.regionserver.compactions.CompactSelection:
> Deleting
> > the expired store file by compaction: hdfs://
> >
> hdpmgr001.pse.movenetworks.com:8020/hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/7426f128469242ec8ee09f3965fd5a1awhosemaxTimeStamp is -1 while the max expired timestamp is 1371398640548
> > >
> > > maxTimeStamp = -1
> > >
> > >
> > > Each time there is a comparison between maxTimeStamp for this store
> file
> > > and the configured maxExpiredTimeStamp and since maxTimeStamp returns
> -1,
> > > it's always elected for a compaction. Now, we need to find why...
> > >
> > > JM
> > >
> > >
> > > 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
> > >
> > >> My cluster is fully distributed (2 regionserver nodes).
> > >>
> > >> Here is a snippet of log entries that may explain why it started:
> > >> http://pastebin.com/wQECif8k. I had to go back 2 days to find when it
> > >> started for this region.
> > >>
> > >> This is not the only region experiencing this issue (but this is the
> > >> smallest one it's happened to).
> > >>
> > >> --Tom
> > >>
> > >>
> > >> On Tue, Sep 24, 2013 at 10:13 AM, Jean-Marc Spaggiari <
> > >> [EMAIL PROTECTED]> wrote:
> > >>
> > >> > Can you past logs a bit before that? To see if anything triggered
> the
> > >> > compaction?
> > >> > Before the 1M compactions entries.
> > >> >
> > >> > Also, what is your setup? Are you running in Standalone?
> Pseudo-Dist?
> > >> > Fully-Dist?
> > >> >
> > >> > Thanks,
> > >> >
> > >> > JM
> > >> >
> > >> >
> > >> > 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
> > >> >
> > >> > > There is one column family, d. Each row has about 10 columns, and
> > each
> > >> > > row's total data size is less than 2K.
> > >> > >
> > >> > > Here is a small snippet of logs from the region server:
> > >> > > http://pastebin.com/S2jE4ZAx
> > >> > >
> > >> > > --Tom
> > >> > >
> > >> > >
> > >> > > On Tue, Sep 24, 2013 at 9:59 AM, Bharath Vissapragada <
> > >> > > [EMAIL PROTECTED]
> > >> > > > wrote:
> > >> > >
> > >> > > > It would help if you can show your RS log (via pastebin?) . Are
> > >> there
> > >> > > > frequent flushes for this region too?
> > >> > > >
> > >> > > >
> > >> > > > On Tue, Sep 24, 2013 at 9:20 PM, Tom Brown <
> [EMAIL PROTECTED]>
> > >> > wrote:
> > >> > > >
> > >> > > > > I have a region that is very small, only 5MB. Despite it's
> size,
> > >> it
> > >> > has
> > >> > > > 24
> > >> > > > > store files. The logs show that it's compacting (over and over
> > >> > again).
> > >> > > > >
> > >> > > > > The odd thing is that even though there are 24 store files, it
> > >> only
> > >> > > does