|
|
-
Hbase memstore manual flush
Shengjie Min 2012-12-04, 15:09
According to Hbase design, Memstore flush is happened automatically behind the theme when it reaches the size limit.
In my case, I want to do a hdfs migration, migrating hdfs data from one cluster to another using distcp, I need to make sure there is nothing left in-memory before I bring down hbase process in the source cluster. Is there anyway we can manually force the flush even tho the memstore hasn't reached the limit. Or, stopping hbase will trigger the flush automatically? if this is the case, how do I know everything in-memory is flushed successfully to HDFS? via metrics(memstoreSize)? -- All the best, Shengjie Min
+
Shengjie Min 2012-12-04, 15:09
-
Re: Hbase memstore manual flush
ramkrishna vasudevan 2012-12-04, 15:22
HBaseAdmin has a flush() api that accepts the table name. This will ensure that the data in memory is flushed.
Regards Ram
On Tue, Dec 4, 2012 at 8:39 PM, Shengjie Min <[EMAIL PROTECTED]> wrote:
> According to Hbase design, Memstore flush is happened automatically behind > the theme when it reaches the size limit. > > In my case, I want to do a hdfs migration, migrating hdfs data from one > cluster to another using distcp, I need to make sure there is nothing left > in-memory before I bring down hbase process in the source cluster. Is there > anyway we can manually force the flush even tho the memstore hasn't reached > the limit. Or, stopping hbase will trigger the flush automatically? if this > is the case, how do I know everything in-memory is flushed successfully to > HDFS? via metrics(memstoreSize)? > -- > All the best, > Shengjie Min >
+
ramkrishna vasudevan 2012-12-04, 15:22
-
Re: Hbase memstore manual flush
Kevin O'dell 2012-12-04, 15:23
>From the master node hbase master stop should also flush everything for shutdown.
On Tue, Dec 4, 2012 at 10:22 AM, ramkrishna vasudevan < [EMAIL PROTECTED]> wrote:
> HBaseAdmin has a flush() api that accepts the table name. This will ensure > that the data in memory is flushed. > > Regards > Ram > > On Tue, Dec 4, 2012 at 8:39 PM, Shengjie Min <[EMAIL PROTECTED]> wrote: > > > According to Hbase design, Memstore flush is happened automatically > behind > > the theme when it reaches the size limit. > > > > In my case, I want to do a hdfs migration, migrating hdfs data from one > > cluster to another using distcp, I need to make sure there is nothing > left > > in-memory before I bring down hbase process in the source cluster. Is > there > > anyway we can manually force the flush even tho the memstore hasn't > reached > > the limit. Or, stopping hbase will trigger the flush automatically? if > this > > is the case, how do I know everything in-memory is flushed successfully > to > > HDFS? via metrics(memstoreSize)? > > -- > > All the best, > > Shengjie Min > > >
-- Kevin O'Dell Customer Operations Engineer, Cloudera
+
Kevin O'dell 2012-12-04, 15:23
|
|