Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> HBase without compactions?


Copy link to this message
-
Re: HBase without compactions?
If you store data in LSM trees you need compactions.
The advantage is that your data files are immutable.
MapR has a mutable file system and they probably store their data in something more akin to B-Trees...?
Or maybe they somehow avoid the expensive merge sorting of many small files. It seems that is has to be one or the other.

(Maybe somebody from MapR reads this and can explain how it actually works.)

Compations let you trade random IO for sequential IO (just to state the obvious). It seems that you can't have it both ways.

-- Lars

________________________________
 From: Otis Gospodnetic <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Monday, February 18, 2013 7:30 PM
Subject: HBase without compactions?
 
Hello,

It's kind of funny, we run SPM, which includes SPM for HBase (performance
monitoring service/tool for HBase essentially) and we currently store all
performance metrics in HBase.

I see a ton of HBase development activity, which is great, but it just
occurred to me that I don't think I recall seeing anything about getting
rid of compactions.  Yet, compactions are one thing that I know hurt us the
most and is one thing that MapR somehow got rid of in their implementation.

Have there been any discussions,attempts, or thoughts about finding a way
to avoid compactions?

Thanks,
Otis
--
HBASE Performance Monitoring - http://sematext.com/spm/index.html
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB