Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Why is this region compacting?


Copy link to this message
-
Re: Why is this region compacting?
Jean-Marc Spaggiari 2013-09-24, 18:10
We get -1 because of this:

      byte [] timerangeBytes = metadataMap.get(TIMERANGE_KEY);
      if (timerangeBytes != null) {
        this.reader.timeRangeTracker = new TimeRangeTracker();
        Writables.copyWritable(timerangeBytes,
this.reader.timeRangeTracker);
      }
this.reader.timeRangeTracker will return -1 for the maximumTimestamp value.
So now, we need to figure if it's normal or not to have TIMERANGE_KEY not
null here.

I have created the same table locally on 0,94.10 with the same attributes
and I'm not facing this issue.

We need to look at the related HFile, but files are rolled VERY quickly, so
might be difficult to get one.

Maybe something like
hadoop fs -get hdfs://
hdpmgr001.pse.movenetworks.com:8020/hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/*
.

might help to get the file? Then we can start to look at it and see what
exactly trigger this behaviour?

JM
2013/9/24 Sergey Shelukhin <[EMAIL PROTECTED]>

> Yeah, I think c3580bdb62d64e42a9eeac50f1c582d2 store file is a good
> example.
> Can you grep for c3580bdb62d64e42a9eeac50f1c582d2 and post the log just to
> be sure? Thanks.
> It looks like an interaction of deleting expired files and
>           // Create the writer even if no kv(Empty store file is also ok),
>           // because we need record the max seq id for the store file, see
>           // HBASE-6059
> in compactor.
> The newly created file is immediately collected the same way and replaced
> by another file, which seems like not an intended behavior, even though
> both pieces of code are technically correct (the empty file is expired, and
> the new file is generally needed).
>
> I filed HBASE-9648
>
>
> On Tue, Sep 24, 2013 at 10:55 AM, Sergey Shelukhin
> <[EMAIL PROTECTED]>wrote:
>
> > To mitigate, you can change hbase.store.delete.expired.storefile to false
> > on one region server, or for entire table, and restart this RS.
> > This will trigger a different compaction, hopefully.
> > We'd need to find what the bug is. My gut feeling (which is known to be
> > wrong often) is that it has to do with it selecting one file, probably
> > invalid check somewhere, or interaction with the code that ensures that
> at
> > least one file needs to be written to preserve metadata, it might be just
> > cycling thru such files.
> >
> >
> > On Tue, Sep 24, 2013 at 10:20 AM, Jean-Marc Spaggiari <
> > [EMAIL PROTECTED]> wrote:
> >
> >> So. Looking at the code, this, for me, sound like a bug.
> >>
> >> I will try to reproduce it locally. Seems to be related to the
> combination
> >> of TTL + BLOOM.
> >>
> >> Creating a table for that right now, will keep you posted very shortly.
> >>
> >> JM
> >>
> >>
> >> 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
> >>
> >> > -rw-------   1 hadoop supergroup       2194 2013-09-21 14:32
> >> >
> >> >
> >>
> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/014ead47a9484d67b55205be16802ff1
> >> > -rw-------   1 hadoop supergroup      31321 2013-09-24 05:49
> >> >
> >> >
> >>
> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/1305d625bd4a4be39a98ae4d91a66140
> >> > -rw-------   1 hadoop supergroup       1350 2013-09-24 10:31
> >> >
> >> >
> >>
> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/1352e0828f974f08b1f3d7a9dff04abd
> >> > -rw-------   1 hadoop supergroup       4194 2013-09-21 10:38
> >> >
> >> >
> >>
> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/17a546064bd840619816809ae0fc4c49
> >> > -rw-------   1 hadoop supergroup       1061 2013-09-20 22:55
> >> >
> >> >
> >>
> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/1cb3df115da244288bd076968ab4ccf6
> >> > -rw-------   1 hadoop supergroup       1375 2013-08-24 10:17
> >> >
> >> >
> >>
> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/1e41a96c49fc4e5ab59392d26935978d
> >> > -rw-------   1 hadoop supergroup      96296 2013-08-26 15:48
> >> >
> >> >
> >>
> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/22d72fd897e34424b5420a96483a571e
> >> > -rw-------   1 hadoop supergroup       1356 2013-08-26 15:23