Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> Best way to compact a region after a move?


+
Jean-Marc Spaggiari 2012-12-30, 16:25
Copy link to this message
-
Re: Best way to compact a region after a move?
balancerCluster() executes on master. Compaction is region server activity.
So they don't pair naturally.

I answered first part of the question in the thread titled 'How to know
it's time for a major compaction?':

In RegionObserver, we already have the following hook:

  /**
   * Called after the region is reported as open to the master.
   * @param c the environment provided by the region server
   */
  void postOpen(final ObserverContext<
RegionCoprocessorEnvironment> c);

Auto-compaction logic can be triggered through the above hook.

Take a look at the following hook for the second part of your question:

  void postCompact(final ObserverContext<RegionCoprocessorEnvironment> c,
final HStore store,
      StoreFile resultFile) throws IOException;

Cheers

On Sun, Dec 30, 2012 at 8:25 AM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Hi,
>
> When I'm balancing manually the regions on my cluster, and I want to
> make sure they are local, so I want to major_compact them each time
> I'm moving them.
>
> On the balanceCluster method, we are returning a list of region to
> move. Which mean they are not yet moved, so I can't compact them
> there.
>
> Is there a place where I shoud hook to compact those regions?
>
> So far, the only idea I found was to start a thread on the
> balancerCluster, wait 1 minute, and compact all the regions I
> returned. But I'm wondering if there is a better way to achieve that?
> Is there a queue where I should place those regions to compact
> instead? Also, I need to know (even if it's just in the logs) when
> those compactions are done.
>
> Thanks,
>
> JM
>
+
Jean-Marc Spaggiari 2012-12-30, 17:47
+
Ted Yu 2012-12-30, 18:11
+
Jean-Marc Spaggiari 2012-12-30, 18:25
+
Ted Yu 2012-12-30, 18:28
+
Jean-Marc Spaggiari 2012-12-30, 18:38
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB