Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> HBase without compactions?

Otis Gospodnetic 2013-02-19, 03:30
Stack 2013-02-19, 06:09
Michael Segel 2013-02-19, 06:47
Andrew Purtell 2013-02-19, 07:12
Copy link to this message
Re: HBase without compactions?
If you store data in LSM trees you need compactions.
The advantage is that your data files are immutable.
MapR has a mutable file system and they probably store their data in something more akin to B-Trees...?
Or maybe they somehow avoid the expensive merge sorting of many small files. It seems that is has to be one or the other.

(Maybe somebody from MapR reads this and can explain how it actually works.)

Compations let you trade random IO for sequential IO (just to state the obvious). It seems that you can't have it both ways.

-- Lars

 From: Otis Gospodnetic <[EMAIL PROTECTED]>
Sent: Monday, February 18, 2013 7:30 PM
Subject: HBase without compactions?

It's kind of funny, we run SPM, which includes SPM for HBase (performance
monitoring service/tool for HBase essentially) and we currently store all
performance metrics in HBase.

I see a ton of HBase development activity, which is great, but it just
occurred to me that I don't think I recall seeing anything about getting
rid of compactions.  Yet, compactions are one thing that I know hurt us the
most and is one thing that MapR somehow got rid of in their implementation.

Have there been any discussions,attempts, or thoughts about finding a way
to avoid compactions?

HBASE Performance Monitoring - http://sematext.com/spm/index.html
Enis Söztutar 2013-02-22, 01:15
Ted Yu 2013-02-19, 04:54
Michael Segel 2013-02-19, 05:46
Otis Gospodnetic 2013-02-19, 05:06
Ted Yu 2013-02-19, 05:09
Michael Segel 2013-02-19, 04:50
Otis Gospodnetic 2013-02-19, 05:01
Michael Segel 2013-02-19, 05:46
Otis Gospodnetic 2013-02-19, 06:05