Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Why is this region compacting?


Copy link to this message
-
Re: Why is this region compacting?
1. Hadoop version is 1.1.2.
2. All servers are synched with NTP.
3. Table definition is: 'compound0', {
NAME => 'd',
DATA_BLOCK_ENCODING => 'NONE',
BLOOMFILTER => 'ROW',
REPLICATION_SCOPE => '0',
VERSIONS => '1',
COMPRESSION => 'SNAPPY',
MIN_VERSIONS => '0',
TTL => '8640000',
KEEP_DELETED_CELLS => 'false',
BLOCKSIZE => '65536',
IN_MEMORY => 'false',
ENCODE_ON_DISK => 'true',
BLOCKCACHE => 'true'
}

The TTL is supposed to be 100 days.

--Tom
On Tue, Sep 24, 2013 at 10:53 AM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Another important information why might be the root cause of this issue...
>
> Do you have any TTL defines for this table?
>
> JM
>
>
> 2013/9/24 Jean-Marc Spaggiari <[EMAIL PROTECTED]>
>
> > Strange.
> >
> > Few questions then.
> > 1) What is your hadoop version?
> > 2) Is the clock on all your severs synched with NTP?
> > 3) What is you table definition? Bloom filters, etc.?
> >
> > This is the reason why it keep compacting:
> >
> > 2013-09-24 10:04:00,548 INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactSelection: Deleting
> the expired store file by compaction: hdfs://
> hdpmgr001.pse.movenetworks.com:8020/hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/7426f128469242ec8ee09f3965fd5a1awhose maxTimeStamp is -1 while the max expired timestamp is 1371398640548
> >
> > maxTimeStamp = -1
> >
> >
> > Each time there is a comparison between maxTimeStamp for this store file
> > and the configured maxExpiredTimeStamp and since maxTimeStamp returns -1,
> > it's always elected for a compaction. Now, we need to find why...
> >
> > JM
> >
> >
> > 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
> >
> >> My cluster is fully distributed (2 regionserver nodes).
> >>
> >> Here is a snippet of log entries that may explain why it started:
> >> http://pastebin.com/wQECif8k. I had to go back 2 days to find when it
> >> started for this region.
> >>
> >> This is not the only region experiencing this issue (but this is the
> >> smallest one it's happened to).
> >>
> >> --Tom
> >>
> >>
> >> On Tue, Sep 24, 2013 at 10:13 AM, Jean-Marc Spaggiari <
> >> [EMAIL PROTECTED]> wrote:
> >>
> >> > Can you past logs a bit before that? To see if anything triggered the
> >> > compaction?
> >> > Before the 1M compactions entries.
> >> >
> >> > Also, what is your setup? Are you running in Standalone? Pseudo-Dist?
> >> > Fully-Dist?
> >> >
> >> > Thanks,
> >> >
> >> > JM
> >> >
> >> >
> >> > 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
> >> >
> >> > > There is one column family, d. Each row has about 10 columns, and
> each
> >> > > row's total data size is less than 2K.
> >> > >
> >> > > Here is a small snippet of logs from the region server:
> >> > > http://pastebin.com/S2jE4ZAx
> >> > >
> >> > > --Tom
> >> > >
> >> > >
> >> > > On Tue, Sep 24, 2013 at 9:59 AM, Bharath Vissapragada <
> >> > > [EMAIL PROTECTED]
> >> > > > wrote:
> >> > >
> >> > > > It would help if you can show your RS log (via pastebin?) . Are
> >> there
> >> > > > frequent flushes for this region too?
> >> > > >
> >> > > >
> >> > > > On Tue, Sep 24, 2013 at 9:20 PM, Tom Brown <[EMAIL PROTECTED]>
> >> > wrote:
> >> > > >
> >> > > > > I have a region that is very small, only 5MB. Despite it's size,
> >> it
> >> > has
> >> > > > 24
> >> > > > > store files. The logs show that it's compacting (over and over
> >> > again).
> >> > > > >
> >> > > > > The odd thing is that even though there are 24 store files, it
> >> only
> >> > > does
> >> > > > > one at a time. Even more strange is that my logs are filling up
> >> with
> >> > > > > compacting this one region. In the last 10 hours, there have
> been
> >> > > > 1,876,200
> >> > > > > log entries corresponding to compacting this region alone.
> >> > > > >
> >> > > > > My cluster is 0.94.10, and using almost all default settings.
> >> Only a
> >> > > few
> >> > > > > are not default:
> >> > > > > hbase.hregion.max.filesize = 4294967296
> >> > > > > hbase.hstore.compaction.min = 6
> >> > > > >
> >> > > > > I am at a total loss as to why this behavior is occurring. Any