Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Accumulo >> mail # user >> How Do I Pragmatically Know When A Compaction Is Over? (i.e., how do I find hotspots)


+
David Medinets 2013-01-02, 15:12
Copy link to this message
-
Re: How Do I Pragmatically Know When A Compaction Is Over? (i.e., how do I find hotspots)
On Wed, Jan 2, 2013 at 7:12 AM, David Medinets <[EMAIL PROTECTED]>wrote:

> I have a map-reduce job which uses AccumuloInputFormat. Some of the
> mappers take 5 minutes while others take 40 minutes. Looking at the
> entry count it seems like some of the tablets have more entries than
> others. I'd like to generate a histogram of the number of entries per
> tablet.
>
> On the way to that goal, I learned that using Bulk Ingest does not
> update the Number of Entries so that I need to perform a compaction
> before I can learn the number of entries... Which leads me to how can
> I tell that a compaction is complete?
>

If you use the "-w" flag to the compact command, the command won't return
until the compaction has completed.  There's also a boolean wait parameter
for one of the TableOperations.compact methods.  Otherwise, you have to
look at the monitor page to see when tablets are no longer being compacted.

Billie
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB