Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Accumulo >> mail # user >> Throttle major compaction.


Copy link to this message
-
Re: Throttle major compaction.
I thnk this could be done fairly easily with an iterator.  An iterator
configured for major compaction could rate limit data.

On Thu, Feb 28, 2013 at 8:27 AM, Denis <[EMAIL PROTECTED]> wrote:
> Hi.
>
> Major compaction loads hard disks very hard, even with
> tserver.compaction.major.concurrent.max=1
>
> Besides nice peaks on the Load Average and IOstat graps during major
> compaction, such high load also badly affects query performance and
> latency.
>
> HBase and Cassandra both have settings to throttle major compaction
> (Cassandra's default is 16 Mb/s which is about 1/3 of the peek values
> I see during Accumulo's major compaction)
>
> But it looks like Accumulo does not have such setting and I have not
> found relevant questions in the bugtracker and maillist archive.
>
> How do you deal with the major compaction and its performance effects?
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB