Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Why is this region compacting?


Copy link to this message
-
Re: Why is this region compacting?
1. Hadoop version is 1.1.2.
2. All servers are synched with NTP.
3. Table definition is: 'compound0', {
NAME => 'd',
DATA_BLOCK_ENCODING => 'NONE',
BLOOMFILTER => 'ROW',
REPLICATION_SCOPE => '0',
VERSIONS => '1',
COMPRESSION => 'SNAPPY',
MIN_VERSIONS => '0',
TTL => '8640000',
KEEP_DELETED_CELLS => 'false',
BLOCKSIZE => '65536',
IN_MEMORY => 'false',
ENCODE_ON_DISK => 'true',
BLOCKCACHE => 'true'
}

The TTL is supposed to be 100 days.

--Tom
On Tue, Sep 24, 2013 at 10:53 AM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Another important information why might be the root cause of this issue...
>
> Do you have any TTL defines for this table?
>
> JM
>
>
> 2013/9/24 Jean-Marc Spaggiari <[EMAIL PROTECTED]>
>
> > Strange.
> >
> > Few questions then.
> > 1) What is your hadoop version?
> > 2) Is the clock on all your severs synched with NTP?
> > 3) What is you table definition? Bloom filters, etc.?
> >
> > This is the reason why it keep compacting:
> >
> > 2013-09-24 10:04:00,548 INFO
> org.apache.hadoop.hbase.regionserver.compactions.CompactSelection: Deleting
> the expired store file by compaction: hdfs://
> hdpmgr001.pse.movenetworks.com:8020/hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/7426f128469242ec8ee09f3965fd5a1awhose maxTimeStamp is -1 while the max expired timestamp is 1371398640548
> >
> > maxTimeStamp = -1
> >
> >
> > Each time there is a comparison between maxTimeStamp for this store file
> > and the configured maxExpiredTimeStamp and since maxTimeStamp returns -1,
> > it's always elected for a compaction. Now, we need to find why...
> >
> > JM
> >
> >
> > 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
> >
> >> My cluster is fully distributed (2 regionserver nodes).
> >>
> >> Here is a snippet of log entries that may explain why it started:
> >> http://pastebin.com/wQECif8k. I had to go back 2 days to find when it
> >> started for this region.
> >>
> >> This is not the only region experiencing this issue (but this is the
> >> smallest one it's happened to).
> >>
> >> --Tom
> >>
> >>
> >> On Tue, Sep 24, 2013 at 10:13 AM, Jean-Marc Spaggiari <
> >> [EMAIL PROTECTED]> wrote:
> >>
> >> > Can you past logs a bit before that? To see if anything triggered the
> >> > compaction?
> >> > Before the 1M compactions entries.
> >> >
> >> > Also, what is your setup? Are you running in Standalone? Pseudo-Dist?
> >> > Fully-Dist?
> >> >
> >> > Thanks,
> >> >
> >> > JM
> >> >
> >> >
> >> > 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
> >> >
> >> > > There is one column family, d. Each row has about 10 columns, and
> each
> >> > > row's total data size is less than 2K.
> >> > >
> >> > > Here is a small snippet of logs from the region server:
> >> > > http://pastebin.com/S2jE4ZAx
> >> > >
> >> > > --Tom
> >> > >
> >> > >
> >> > > On Tue, Sep 24, 2013 at 9:59 AM, Bharath Vissapragada <
> >> > > [EMAIL PROTECTED]
> >> > > > wrote:
> >> > >
> >> > > > It would help if you can show your RS log (via pastebin?) . Are
> >> there
> >> > > > frequent flushes for this region too?
> >> > > >
> >> > > >
> >> > > > On Tue, Sep 24, 2013 at 9:20 PM, Tom Brown <[EMAIL PROTECTED]>
> >> > wrote:
> >> > > >
> >> > > > > I have a region that is very small, only 5MB. Despite it's size,
> >> it
> >> > has
> >> > > > 24
> >> > > > > store files. The logs show that it's compacting (over and over
> >> > again).
> >> > > > >
> >> > > > > The odd thing is that even though there are 24 store files, it
> >> only
> >> > > does
> >> > > > > one at a time. Even more strange is that my logs are filling up
> >> with
> >> > > > > compacting this one region. In the last 10 hours, there have
> been
> >> > > > 1,876,200
> >> > > > > log entries corresponding to compacting this region alone.
> >> > > > >
> >> > > > > My cluster is 0.94.10, and using almost all default settings.
> >> Only a
> >> > > few
> >> > > > > are not default:
> >> > > > > hbase.hregion.max.filesize = 4294967296
> >> > > > > hbase.hstore.compaction.min = 6
> >> > > > >
> >> > > > > I am at a total loss as to why this behavior is occurring. Any
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB