Home | About | Sematext search-lucene.com search-hadoop.com
clear query|facets|time Search criteria: compactions.   Results from 1 to 10 from 17 (0.139s).
Did you mean:
SpellChecker(?): compaction 
Loading phrases to help you
refine your search...
Re: Disabled automated compaction - table still compacting - HBase - [mail # user]
...You should be able to limit what JM describes by tuning the following two configs:  hbase.hstore.compactionThreshold hbase.hstore.compaction.max  Beware of this property as well when tuning...
   Author: Bryan Beaudreault, 2013-07-09, 14:48
Smart Managed Major Compactions - HBase - [mail # user]
...Hello all,   Before I start, I'm running cdh3u2, so 0.90.4.  I am looking into managing major compactions ourselves, but there doesn't appear to be any mechanisms I can hook in to determine...
... which tables need compacting.  Ideally each time my cron job runs it would compact the table with the next longest time since compaction, but I can't find a way to access this metric...
[+ show more]
   Author: Bryan Beaudreault, 2012-07-18, 17:26
Re: How to pick which region(s) to major compact? - HBase - [mail # user]
... with the oldest modified timestamp, and major_compact those.  3) At the end of the day, our servers were not really utilizing 100% of disk and CPU so we decided to just major compact everything each night...
....  We staggered the compactions over a couple hours so as not to overwhelm, but not sure if that has much effect since it is in serial in a single thread anyway.   On Wed, Dec 12, 2012 at 3...
   Author: Bryan Beaudreault, 2012-12-12, 23:46
Bulk loading (and/or major compaction) causing OOM - HBase - [mail # user]
... these in now, but every time we do our servers die with OOM.  The logs seem to show that there is always a major compaction happening when the OOM happens.  This is among other normal usage from...
... a variety of apps in our product, so the memstores, block cache, etc are all active during this time.  I was reading through the compaction code and it doesn't look like it should take up much...
[+ show more]
[+ more]    Author: Bryan Beaudreault, 2012-12-07, 21:01
Re: Full stack rolling restart - HBase - [mail # user]
..., but it's hard to tell.  I think the only truly safe way to do this with CDH4.x is: 1. Move all regions off regionserver 2. Compact all regions after they have moved off 3. Restart DN and RS. 4...
.... Wait for the DN to register with the namenode which is not always super fast. 5. Move everything back. 6. Compact again. This is not great because it will take a while and adds a bunch...
[+ show more]
   Author: Bryan Beaudreault, 2014-05-29, 15:46
Re: hadoop block locations - HBase - [mail # user]
...I responded to your post on the hadoop user list, but you will need to do manual major compactions to recover locality.  It may recover automatically over time, but only...
   Author: Bryan Beaudreault, 2014-03-28, 01:52
Re: 3-Hour Periodic Network/CPU/Disk/Latency Spikes - HBase - [mail # user]
...Have you taken a look at the logs on the RegionServers during the period?  One possibility is compactions happening organically.  If you were sustaining a certain level of writes most...
... of the time, I could maybe see that every 3 hours enough store files build up to require compactions.  There's nothing else automated in HDFS or HBase that I could see causing this.  On Fri, Dec 13...
   Author: Bryan Beaudreault, 2013-12-13, 20:19
Re: experiencing high latency for few reads in HBase - HBase - [mail # user]
...If you lost a RS or otherwise moved a region around (hbase balancer?) then it would throw off the locality.  You'll want to compact any time a region moves for any reason (unless it moved...
... to another RS with which it had locality).  Try major compacting again now and see if the locality index goes up to 100   On Thu, Aug 29, 2013 at 1:52 PM, Saurabh Yahoo  wrote:  ...
   Author: Bryan Beaudreault, 2013-08-29, 17:55
Re: Hbase tuning for heavy write cluster - HBase - [mail # user]
... especially for something like opentsdb I think.  If you are going to up the blockingStoreFiles you're probably also going to want to up hbase.hstore.compaction.max. For my tsdb cluster, which is 8 i...
...2.4xlarges in EC2, we have 90 regions for tsdb.  We were also having issues with blocking, and I upped blockingStoreFiles to 35, compaction.max to 15, and memstore.block.multiplier to 3.  We...
[+ show more]
[+ more]    Author: Bryan Beaudreault, 2014-01-25, 02:51
Re: Practical Upper Limit on Number of Version Stored? - HBase - [mail # user]
... up on major compaction.  This row eventually grew to the size of a region and became an operational nightmare, because you can't split inside the row boundary and other bugs around...
   Author: Bryan Beaudreault, 2013-12-06, 02:28
Sort:
project
HBase (17)
HDFS (1)
Hadoop (1)
type
mail # user (17)
mail # dev (1)
date
last 7 days (0)
last 30 days (0)
last 90 days (1)
last 6 months (2)
last 9 months (17)
author
Jean-Daniel Cryans (58)
Stack (55)
Ted Yu (31)
Jean-Marc Spaggiari (28)
lars hofhansl (20)
Ryan Rawson (15)
Andrew Purtell (12)
Doug Meil (12)
Michael Segel (11)
Vidhyashankar Venkatarama...(10)
yun peng (8)
Alex Baranau (7)
Asaf Mesika (7)
Bryan Beaudreault (7)
Buttler, David (7)
Harsh J (7)
Jack Levin (7)
Jinsong Hu (7)
Matt Corgan (7)
Amit Sela (6)
Dmitriy Lyubimov (6)
Jonathan Gray (6)
Lars George (6)
Otis Gospodnetic (6)
Varun Sharma (6)