Home | About | Sematext search-lucene.com search-hadoop.com
clear query|facets|time Search criteria: compactions.   Results from 1 to 10 from 20 (0.195s).
Did you mean:
SpellChecker(?): compaction 
Loading phrases to help you
refine your search...
Re: Disabled automated compaction - table still compacting - HBase - [mail # user]
...You should be able to limit what JM describes by tuning the following two configs:  hbase.hstore.compactionThreshold hbase.hstore.compaction.max  Beware of this property as well when tuning...
   Author: Bryan Beaudreault, 2013-07-09, 14:48
[expand - 1 more] - Re: GC peaks during major compaction - HBase - [mail # user]
...You're gonna want to use java7 and the G1 collector with a heap that high. I'm surprised you aren't seeing other issues. It's possible that with the added CPU load from the compactions...
   Author: Bryan Beaudreault, 2014-08-20, 10:11
[expand - 1 more] - Re: Non-sensical? Compaction Metrics Hbase UI - HBase - [mail # user]
...I've also seen that on 0.94.x.  The compaction progress is almost never accurate. I've taken to doing: hdfs -du /hbase///.tmp to track the new hfile size, expecting...
... it to be approximately the same size as total hfiles pre-compaction. On Mon, Aug 11, 2014 at 3:11 PM, Geovanie Marquez < [EMAIL PROTECTED]> wrote:   ...
   Author: Bryan Beaudreault, 2014-08-11, 19:15
Smart Managed Major Compactions - HBase - [mail # user]
...Hello all,   Before I start, I'm running cdh3u2, so 0.90.4.  I am looking into managing major compactions ourselves, but there doesn't appear to be any mechanisms I can hook in to determine...
... which tables need compacting.  Ideally each time my cron job runs it would compact the table with the next longest time since compaction, but I can't find a way to access this metric...
[+ show more]
   Author: Bryan Beaudreault, 2012-07-18, 17:26
Re: How to pick which region(s) to major compact? - HBase - [mail # user]
... with the oldest modified timestamp, and major_compact those.  3) At the end of the day, our servers were not really utilizing 100% of disk and CPU so we decided to just major compact everything each night...
....  We staggered the compactions over a couple hours so as not to overwhelm, but not sure if that has much effect since it is in serial in a single thread anyway.   On Wed, Dec 12, 2012 at 3...
   Author: Bryan Beaudreault, 2012-12-12, 23:46
[expand - 2 more] - Bulk loading (and/or major compaction) causing OOM - HBase - [mail # user]
... these in now, but every time we do our servers die with OOM.  The logs seem to show that there is always a major compaction happening when the OOM happens.  This is among other normal usage from...
... a variety of apps in our product, so the memstores, block cache, etc are all active during this time.  I was reading through the compaction code and it doesn't look like it should take up much...
[+ show more]
   Author: Bryan Beaudreault, 2012-12-07, 21:01
Re: state-of-the-art method for merging regions on v0.94 - HBase - [mail # user]
... hbase code and doing tests on a test cluster) of how it all works.  And even then I felt nervous to do it in prod.  Hence why I went the snapshot/compact route. I would definitely test...
   Author: Bryan Beaudreault, 2014-08-28, 18:27
Re: Full stack rolling restart - HBase - [mail # user]
..., but it's hard to tell.  I think the only truly safe way to do this with CDH4.x is: 1. Move all regions off regionserver 2. Compact all regions after they have moved off 3. Restart DN and RS. 4...
.... Wait for the DN to register with the namenode which is not always super fast. 5. Move everything back. 6. Compact again. This is not great because it will take a while and adds a bunch...
[+ show more]
   Author: Bryan Beaudreault, 2014-05-29, 15:46
Re: hadoop block locations - HBase - [mail # user]
...I responded to your post on the hadoop user list, but you will need to do manual major compactions to recover locality.  It may recover automatically over time, but only...
   Author: Bryan Beaudreault, 2014-03-28, 01:52
Re: 3-Hour Periodic Network/CPU/Disk/Latency Spikes - HBase - [mail # user]
...Have you taken a look at the logs on the RegionServers during the period?  One possibility is compactions happening organically.  If you were sustaining a certain level of writes most...
... of the time, I could maybe see that every 3 hours enough store files build up to require compactions.  There's nothing else automated in HDFS or HBase that I could see causing this.  On Fri, Dec 13...
   Author: Bryan Beaudreault, 2013-12-13, 20:19
Sort:
project
HBase (20)
HDFS (1)
Hadoop (1)
type
mail # user (20)
mail # dev (2)
date
last 7 days (0)
last 30 days (2)
last 90 days (3)
last 6 months (5)
last 9 months (20)
author
Jean-Daniel Cryans (57)
Stack (55)
Ted Yu (34)
Jean-Marc Spaggiari (30)
lars hofhansl (20)
Ryan Rawson (15)
Andrew Purtell (12)
Doug Meil (12)
Michael Segel (11)
Vidhyashankar Venkatarama...(10)
Bryan Beaudreault (8)
yun peng (8)
Alex Baranau (7)
Asaf Mesika (7)
Buttler, David (7)
Harsh J (7)
Jack Levin (7)
Jinsong Hu (7)
Matt Corgan (7)
Amit Sela (6)
Dmitriy Lyubimov (6)
Jonathan Gray (6)
Lars George (6)
Otis Gospodnetic (6)
Varun Sharma (6)