Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Why is this region compacting?


Copy link to this message
-
Re: Why is this region compacting?
Yeah, I think c3580bdb62d64e42a9eeac50f1c582d2 store file is a good example.
Can you grep for c3580bdb62d64e42a9eeac50f1c582d2 and post the log just to
be sure? Thanks.
It looks like an interaction of deleting expired files and
          // Create the writer even if no kv(Empty store file is also ok),
          // because we need record the max seq id for the store file, see
          // HBASE-6059
in compactor.
The newly created file is immediately collected the same way and replaced
by another file, which seems like not an intended behavior, even though
both pieces of code are technically correct (the empty file is expired, and
the new file is generally needed).

I filed HBASE-9648
On Tue, Sep 24, 2013 at 10:55 AM, Sergey Shelukhin
<[EMAIL PROTECTED]>wrote:

> To mitigate, you can change hbase.store.delete.expired.storefile to false
> on one region server, or for entire table, and restart this RS.
> This will trigger a different compaction, hopefully.
> We'd need to find what the bug is. My gut feeling (which is known to be
> wrong often) is that it has to do with it selecting one file, probably
> invalid check somewhere, or interaction with the code that ensures that at
> least one file needs to be written to preserve metadata, it might be just
> cycling thru such files.
>
>
> On Tue, Sep 24, 2013 at 10:20 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> So. Looking at the code, this, for me, sound like a bug.
>>
>> I will try to reproduce it locally. Seems to be related to the combination
>> of TTL + BLOOM.
>>
>> Creating a table for that right now, will keep you posted very shortly.
>>
>> JM
>>
>>
>> 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
>>
>> > -rw-------   1 hadoop supergroup       2194 2013-09-21 14:32
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/014ead47a9484d67b55205be16802ff1
>> > -rw-------   1 hadoop supergroup      31321 2013-09-24 05:49
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/1305d625bd4a4be39a98ae4d91a66140
>> > -rw-------   1 hadoop supergroup       1350 2013-09-24 10:31
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/1352e0828f974f08b1f3d7a9dff04abd
>> > -rw-------   1 hadoop supergroup       4194 2013-09-21 10:38
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/17a546064bd840619816809ae0fc4c49
>> > -rw-------   1 hadoop supergroup       1061 2013-09-20 22:55
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/1cb3df115da244288bd076968ab4ccf6
>> > -rw-------   1 hadoop supergroup       1375 2013-08-24 10:17
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/1e41a96c49fc4e5ab59392d26935978d
>> > -rw-------   1 hadoop supergroup      96296 2013-08-26 15:48
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/22d72fd897e34424b5420a96483a571e
>> > -rw-------   1 hadoop supergroup       1356 2013-08-26 15:23
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/25fee1ffadbe42549bd0b7b13d782b72
>> > -rw-------   1 hadoop supergroup       6229 2013-09-21 11:14
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/26289c777ec14dc5b7021b4d6b1050c5
>> > -rw-------   1 hadoop supergroup       1223 2013-09-21 02:42
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/2757d7ba9c8448d6a3d5d46bd4d59758
>> > -rw-------   1 hadoop supergroup    5302248 2013-08-24 02:22
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/2ec40943787246ea983608dd6591db24
>> > -rw-------   1 hadoop supergroup       1596 2013-08-24 03:37
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/3157fd1cabe4483aaa4d9a21f75e4d88
>> > -rw-------   1 hadoop supergroup       1338 2013-09-22 04:25
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/36b0f80a4a7b492f97358b64d879a2df
>> > -rw-------   1 hadoop supergroup       3264 2013-09-21 12:05
>> >
>> >
>> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/39e249fcb532400daed73aed6689ceeb
>> > -rw-------   1 hadoop supergroup       4549 2013-09-21 08:56

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB