Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Re: Rebuild HBASE Table to reduce Regions per RS


+
Upender Nimbekar 2014-01-14, 20:38
Copy link to this message
-
Re: Rebuild HBASE Table to reduce Regions per RS

Normally when people are doing merges they are scripting it (I.E., not
typing out or copying/pasting each region name), but if a table is that
far gone the least problematic way is to re-create assuming you have the
capacity (i.e., not just storage, but nodes to support all the extra hbase
regions before dropping the old table).

On 1/14/14 3:38 PM, "Upender Nimbekar" <[EMAIL PROTECTED]> wrote:

>Is there a way I can Merge w/o explicitly mentioning the region names. I
>have got a total of 50000 regions. There's no way I am going to name each
>and every region. Otherway is to MIRROR a new table. Use Map Reduce job to
>copy data from old table. Delete Old Table. And Rename new table. What do
>you think. I was almost inclined to used MERGE but looks like that's not
>the way.
>
>Thanks
>Upender
>
>
>On Tue, Jan 14, 2014 at 1:47 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> Upender:
>> For 15.2.2 Merge, please note the following condition:
>>
>>     LOG.info("Verifying that HBase is not running...");
>>     try {
>>       HBaseAdmin.checkHBaseAvailable(getConf());
>>       LOG.fatal("HBase cluster must be off-line.");
>>
>> Cheers
>>
>>
>> On Tue, Jan 14, 2014 at 10:40 AM, Vladimir Rodionov
>> <[EMAIL PROTECTED]>wrote:
>>
>> > I have never tried this before but I think the following should work:
>> >
>> > 1. Alter your table:
>> >
>> > habse> alter 't1', METHOD => 'table_att', MAX_FILESIZE => '5000000000'
>> > (place your own number here)
>> >
>> > 2. Merge regions:
>> >
>> > http://hbase.apache.org/book/ops.regionmgt.html
>> >
>> >
>> >
>> >
>> > On Tue, Jan 14, 2014 at 7:21 AM, Upender Nimbekar <
>> > [EMAIL PROTECTED]> wrote:
>> >
>> > > Hi,
>> > > Does anyone have any experience rebuidling the HBASE table to reduce
>> the
>> > > number of regions. I am currently dealing with a situation where the
>> no.
>> > of
>> > > regions per RS have gone up quite significantly (500 per RS) and
>> thereby
>> > > causing some performance issues. This is how I am thinking of
>>bringing
>> it
>> > > down:
>> > >
>> > > increase the hbase.hregion.max.filesize from 500 MB to 2 GB
>> > >
>> > > And the rebuild the HBASE table. I am assuming if after table
>>rebuild,
>> I
>> > > should see the no. of regions come down to more than half. I would
>> > > basically like to stay within HBASE suggested no. of regions per RS
>> which
>> > > is about 50-200.
>> > >
>> > > Please suggest if someone has any experience doing it.
>> > >
>> > > Thanks
>> > > Upen
>> > >
>> >
>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB