Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Why is this region compacting?


Copy link to this message
-
Re: Why is this region compacting?
Meanwhile you can mitigate as specified above, by temporarily disabling
expired file deletion.
Please report if it doesn't work...
On Tue, Sep 24, 2013 at 4:08 PM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi Tom,
>
> Thanks for reporting this and for providing all this information.
>
> I have attached a patch on the JIRA that Sergey's opened.
>
> This will need to be reviewed and we will need a commiter to push it if
> it's accepted.
>
> JM
>
>
> 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
>
> > I tried the workaround, and it is working very well. The number of store
> > files for all regions is now sane (went from about 8000 total store files
> > to 1000), and scans are now much more efficient.
> >
> > Thanks for all your help, Jean-Marc and Sergey!
> >
> > --Tom
> >
> >
> > On Tue, Sep 24, 2013 at 2:11 PM, Jean-Marc Spaggiari <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi Tom,
> > >
> > > Thanks for this information and the offer. I think we have enought to
> > start
> > > to look at this issue. I'm still trying to reproduce that locally. In
> the
> > > meantime, I sent a patch to fix the NullPointException your faced
> before.
> > >
> > > I will post back here if I'm able to reproduce. Have you tried Sergey's
> > > workarround?
> > >
> > > JM
> > >
> > >
> > > 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
> > >
> > > > Yes, it is empty.
> > > >
> > > > 13/09/24 13:03:03 INFO hfile.CacheConfig: Allocating LruBlockCache
> with
> > > > maximum size 2.9g
> > > > 13/09/24 13:03:03 ERROR metrics.SchemaMetrics: Inconsistent
> > > configuration.
> > > > Previous configuration for using table name in metrics: true, new
> > > > configuration: false
> > > > 13/09/24 13:03:03 WARN metrics.SchemaConfigured: Could not determine
> > > table
> > > > and column family of the HFile path
> /fca0882dc7624342a8f4fce4b89420ff.
> > > > Expecting at least 5 path components.
> > > > 13/09/24 13:03:03 WARN snappy.LoadSnappy: Snappy native library is
> > > > available
> > > > 13/09/24 13:03:03 INFO util.NativeCodeLoader: Loaded the
> native-hadoop
> > > > library
> > > > 13/09/24 13:03:03 INFO snappy.LoadSnappy: Snappy native library
> loaded
> > > > 13/09/24 13:03:03 INFO compress.CodecPool: Got brand-new decompressor
> > > > Stats:
> > > > no data available for statistics
> > > > Scanned kv count -> 0
> > > >
> > > > If you want to examine the actual file, I would be happy to email it
> to
> > > you
> > > > directly.
> > > >
> > > > --Tom
> > > >
> > > >
> > > > On Tue, Sep 24, 2013 at 12:42 PM, Jean-Marc Spaggiari <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > > > Can you try with less parameters and see if you are able to get
> > > something
> > > > > from it? This exception is caused by the "printMeta", so if you
> > remove
> > > -m
> > > > > it should be ok. However, printMeta was what I was looking for ;)
> > > > >
> > > > > getFirstKey for this file seems to return null. So it might simply
> be
> > > an
> > > > > empty file, not necessary a corrupted one.
> > > > >
> > > > >
> > > > > 2013/9/24 Tom Brown <[EMAIL PROTECTED]>
> > > > >
> > > > > > /usr/lib/hbase/bin/hbase org.apache.hadoop.hbase.io.hfile.HFile
> -m
> > -s
> > > > -v
> > > > > -f
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/fca0882dc7624342a8f4fce4b89420ff
> > > > > > 13/09/24 12:33:40 INFO util.ChecksumType: Checksum using
> > > > > > org.apache.hadoop.util.PureJavaCrc32
> > > > > > Scanning ->
> > > > > >
> > > > > >
> > > > >
> > > >
> > >
> >
> /hbase/compound3/5ab5fdfcf2aff2633e1d6d5089c96aa2/d/fca0882dc7624342a8f4fce4b89420ff
> > > > > > 13/09/24 12:33:41 INFO hfile.CacheConfig: Allocating
> LruBlockCache
> > > with
> > > > > > maximum size 2.9g
> > > > > > 13/09/24 12:33:41 ERROR metrics.SchemaMetrics: Inconsistent
> > > > > configuration.
> > > > > > Previous configuration for using table name in metrics: true, new
> > > > > > configuration: false
> > > > > > 13/09/24 12:33:41 WARN snappy.LoadSnappy: Snappy native library

CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB