Schubert Zhang
2011-05-20, 07:25
Ted Yu
2011-05-20, 15:33
Schubert Zhang
2011-05-28, 16:57
Anty
2011-06-01, 02:44
Stack
2011-06-01, 03:02
Schubert Zhang
2011-06-01, 04:13
Anty
2011-06-01, 06:26
Ted Yu
2011-06-01, 16:32
Stack
2011-06-01, 17:25
Anty
2011-06-02, 08:19
Schubert Zhang
2011-06-02, 15:41
-
HBase balancer policy issueSchubert Zhang 2011-05-20, 07:25
I have a question about HBase balancer.
In release 0.90.x, it seems the balancer only regards the number of regions
and balance these regions into every regionserver.
If we have two tables (A and B) now, each have 100 regions.
Then, a extreme situation is:
RegionsServer1: 100 regions, which all belong to table A
RegionsServer2: 100 regions, which all belong to table B
If my application access table B heavy, the almost all opetations hit
RegionsServer2, it is not balance.
I have a idea about the balance policy:
(1) Firstly balance for each table
(2) Then, overall balance.
Schubert
In release 0.90.x, it seems the balancer only regards the number of regions
and balance these regions into every regionserver.
If we have two tables (A and B) now, each have 100 regions.
Then, a extreme situation is:
RegionsServer1: 100 regions, which all belong to table A
RegionsServer2: 100 regions, which all belong to table B
If my application access table B heavy, the almost all opetations hit
RegionsServer2, it is not balance.
I have a idea about the balance policy:
(1) Firstly balance for each table
(2) Then, overall balance.
Schubert
-
Re: HBase balancer policy issueTed Yu 2011-05-20, 15:33
I think your request is described in
https://issues.apache.org/jira/browse/HBASE-3373
Practically speaking, the scenario below can hardly occur (in trunk, at
least).
If the tables are created with pre-split regions, the regions would be
round-robin distributed.
If the tables are created with single region, subsequent write operations
would cause the region split. Balancer would offload young regions to other
servers - see HBASE-3609
<https://issues.apache.org/jira/browse/HBASE-3609>which is not in 0.90
branch.
Refer to
http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.html for
details.
Cheers
On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]> wrote:
> I have a question about HBase balancer.
>
> In release 0.90.x, it seems the balancer only regards the number of regions
> and balance these regions into every regionserver.
>
> If we have two tables (A and B) now, each have 100 regions.
> Then, a extreme situation is:
>
> RegionsServer1: 100 regions, which all belong to table A
> RegionsServer2: 100 regions, which all belong to table B
>
> If my application access table B heavy, the almost all opetations hit
> RegionsServer2, it is not balance.
>
> I have a idea about the balance policy:
> (1) Firstly balance for each table
> (2) Then, overall balance.
>
>
> Schubert
>
https://issues.apache.org/jira/browse/HBASE-3373
Practically speaking, the scenario below can hardly occur (in trunk, at
least).
If the tables are created with pre-split regions, the regions would be
round-robin distributed.
If the tables are created with single region, subsequent write operations
would cause the region split. Balancer would offload young regions to other
servers - see HBASE-3609
<https://issues.apache.org/jira/browse/HBASE-3609>which is not in 0.90
branch.
Refer to
http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.html for
details.
Cheers
On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]> wrote:
> I have a question about HBase balancer.
>
> In release 0.90.x, it seems the balancer only regards the number of regions
> and balance these regions into every regionserver.
>
> If we have two tables (A and B) now, each have 100 regions.
> Then, a extreme situation is:
>
> RegionsServer1: 100 regions, which all belong to table A
> RegionsServer2: 100 regions, which all belong to table B
>
> If my application access table B heavy, the almost all opetations hit
> RegionsServer2, it is not balance.
>
> I have a idea about the balance policy:
> (1) Firstly balance for each table
> (2) Then, overall balance.
>
>
> Schubert
>
-
Re: HBase balancer policy issueSchubert Zhang 2011-05-28, 16:57
Thanks, I think it is HBASE-3373
On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> I think your request is described in
> https://issues.apache.org/jira/browse/HBASE-3373
>
> Practically speaking, the scenario below can hardly occur (in trunk, at
> least).
> If the tables are created with pre-split regions, the regions would be
> round-robin distributed.
> If the tables are created with single region, subsequent write operations
> would cause the region split. Balancer would offload young regions to other
> servers - see HBASE-3609
> <https://issues.apache.org/jira/browse/HBASE-3609>which is not in 0.90
> branch.
>
> Refer to
> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.html for
> details.
>
> Cheers
>
> On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
> wrote:
>
> > I have a question about HBase balancer.
> >
> > In release 0.90.x, it seems the balancer only regards the number of
> regions
> > and balance these regions into every regionserver.
> >
> > If we have two tables (A and B) now, each have 100 regions.
> > Then, a extreme situation is:
> >
> > RegionsServer1: 100 regions, which all belong to table A
> > RegionsServer2: 100 regions, which all belong to table B
> >
> > If my application access table B heavy, the almost all opetations hit
> > RegionsServer2, it is not balance.
> >
> > I have a idea about the balance policy:
> > (1) Firstly balance for each table
> > (2) Then, overall balance.
> >
> >
> > Schubert
> >
>
On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> I think your request is described in
> https://issues.apache.org/jira/browse/HBASE-3373
>
> Practically speaking, the scenario below can hardly occur (in trunk, at
> least).
> If the tables are created with pre-split regions, the regions would be
> round-robin distributed.
> If the tables are created with single region, subsequent write operations
> would cause the region split. Balancer would offload young regions to other
> servers - see HBASE-3609
> <https://issues.apache.org/jira/browse/HBASE-3609>which is not in 0.90
> branch.
>
> Refer to
> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.html for
> details.
>
> Cheers
>
> On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
> wrote:
>
> > I have a question about HBase balancer.
> >
> > In release 0.90.x, it seems the balancer only regards the number of
> regions
> > and balance these regions into every regionserver.
> >
> > If we have two tables (A and B) now, each have 100 regions.
> > Then, a extreme situation is:
> >
> > RegionsServer1: 100 regions, which all belong to table A
> > RegionsServer2: 100 regions, which all belong to table B
> >
> > If my application access table B heavy, the almost all opetations hit
> > RegionsServer2, it is not balance.
> >
> > I have a idea about the balance policy:
> > (1) Firstly balance for each table
> > (2) Then, overall balance.
> >
> >
> > Schubert
> >
>
-
Re: HBase balancer policy issueAnty 2011-06-01, 02:44
When doing balance, Can we take into account the compaction status of
regions.
Currently, even the region is doing compaction, it can also be interrupted
to response to reassign.
On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]> wrote:
> Thanks, I think it is HBASE-3373
>
> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > I think your request is described in
> > https://issues.apache.org/jira/browse/HBASE-3373
> >
> > Practically speaking, the scenario below can hardly occur (in trunk, at
> > least).
> > If the tables are created with pre-split regions, the regions would be
> > round-robin distributed.
> > If the tables are created with single region, subsequent write operations
> > would cause the region split. Balancer would offload young regions to
> other
> > servers - see HBASE-3609
> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in 0.90
> > branch.
> >
> > Refer to
> > http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> > details.
> >
> > Cheers
> >
> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I have a question about HBase balancer.
> > >
> > > In release 0.90.x, it seems the balancer only regards the number of
> > regions
> > > and balance these regions into every regionserver.
> > >
> > > If we have two tables (A and B) now, each have 100 regions.
> > > Then, a extreme situation is:
> > >
> > > RegionsServer1: 100 regions, which all belong to table A
> > > RegionsServer2: 100 regions, which all belong to table B
> > >
> > > If my application access table B heavy, the almost all opetations hit
> > > RegionsServer2, it is not balance.
> > >
> > > I have a idea about the balance policy:
> > > (1) Firstly balance for each table
> > > (2) Then, overall balance.
> > >
> > >
> > > Schubert
> > >
> >
>
--
Best Regards
Anty Rao
regions.
Currently, even the region is doing compaction, it can also be interrupted
to response to reassign.
On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]> wrote:
> Thanks, I think it is HBASE-3373
>
> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > I think your request is described in
> > https://issues.apache.org/jira/browse/HBASE-3373
> >
> > Practically speaking, the scenario below can hardly occur (in trunk, at
> > least).
> > If the tables are created with pre-split regions, the regions would be
> > round-robin distributed.
> > If the tables are created with single region, subsequent write operations
> > would cause the region split. Balancer would offload young regions to
> other
> > servers - see HBASE-3609
> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in 0.90
> > branch.
> >
> > Refer to
> > http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> > details.
> >
> > Cheers
> >
> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
> > wrote:
> >
> > > I have a question about HBase balancer.
> > >
> > > In release 0.90.x, it seems the balancer only regards the number of
> > regions
> > > and balance these regions into every regionserver.
> > >
> > > If we have two tables (A and B) now, each have 100 regions.
> > > Then, a extreme situation is:
> > >
> > > RegionsServer1: 100 regions, which all belong to table A
> > > RegionsServer2: 100 regions, which all belong to table B
> > >
> > > If my application access table B heavy, the almost all opetations hit
> > > RegionsServer2, it is not balance.
> > >
> > > I have a idea about the balance policy:
> > > (1) Firstly balance for each table
> > > (2) Then, overall balance.
> > >
> > >
> > > Schubert
> > >
> >
>
--
Best Regards
Anty Rao
-
Re: HBase balancer policy issueStack 2011-06-01, 03:02
If compacting, don't we interrupt it so we can close and move the
region. You are worried about the compacting work done so far -- you
don't want to lose it? So you are suggesting that a region should be
able to say "No, not now! I'm busy?" (We'd need to distingush between
a balancer 'move' and a move or close for any other region).
St.Ack
On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> When doing balance, Can we take into account the compaction status of
> regions.
> Currently, even the region is doing compaction, it can also be interrupted
> to response to reassign.
>
>
> On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]> wrote:
>
>> Thanks, I think it is HBASE-3373
>>
>> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > I think your request is described in
>> > https://issues.apache.org/jira/browse/HBASE-3373
>> >
>> > Practically speaking, the scenario below can hardly occur (in trunk, at
>> > least).
>> > If the tables are created with pre-split regions, the regions would be
>> > round-robin distributed.
>> > If the tables are created with single region, subsequent write operations
>> > would cause the region split. Balancer would offload young regions to
>> other
>> > servers - see HBASE-3609
>> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in 0.90
>> > branch.
>> >
>> > Refer to
>> > http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
>> > details.
>> >
>> > Cheers
>> >
>> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
>> > wrote:
>> >
>> > > I have a question about HBase balancer.
>> > >
>> > > In release 0.90.x, it seems the balancer only regards the number of
>> > regions
>> > > and balance these regions into every regionserver.
>> > >
>> > > If we have two tables (A and B) now, each have 100 regions.
>> > > Then, a extreme situation is:
>> > >
>> > > RegionsServer1: 100 regions, which all belong to table A
>> > > RegionsServer2: 100 regions, which all belong to table B
>> > >
>> > > If my application access table B heavy, the almost all opetations hit
>> > > RegionsServer2, it is not balance.
>> > >
>> > > I have a idea about the balance policy:
>> > > (1) Firstly balance for each table
>> > > (2) Then, overall balance.
>> > >
>> > >
>> > > Schubert
>> > >
>> >
>>
>
>
>
> --
> Best Regards
> Anty Rao
>
region. You are worried about the compacting work done so far -- you
don't want to lose it? So you are suggesting that a region should be
able to say "No, not now! I'm busy?" (We'd need to distingush between
a balancer 'move' and a move or close for any other region).
St.Ack
On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> When doing balance, Can we take into account the compaction status of
> regions.
> Currently, even the region is doing compaction, it can also be interrupted
> to response to reassign.
>
>
> On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]> wrote:
>
>> Thanks, I think it is HBASE-3373
>>
>> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>>
>> > I think your request is described in
>> > https://issues.apache.org/jira/browse/HBASE-3373
>> >
>> > Practically speaking, the scenario below can hardly occur (in trunk, at
>> > least).
>> > If the tables are created with pre-split regions, the regions would be
>> > round-robin distributed.
>> > If the tables are created with single region, subsequent write operations
>> > would cause the region split. Balancer would offload young regions to
>> other
>> > servers - see HBASE-3609
>> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in 0.90
>> > branch.
>> >
>> > Refer to
>> > http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
>> > details.
>> >
>> > Cheers
>> >
>> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
>> > wrote:
>> >
>> > > I have a question about HBase balancer.
>> > >
>> > > In release 0.90.x, it seems the balancer only regards the number of
>> > regions
>> > > and balance these regions into every regionserver.
>> > >
>> > > If we have two tables (A and B) now, each have 100 regions.
>> > > Then, a extreme situation is:
>> > >
>> > > RegionsServer1: 100 regions, which all belong to table A
>> > > RegionsServer2: 100 regions, which all belong to table B
>> > >
>> > > If my application access table B heavy, the almost all opetations hit
>> > > RegionsServer2, it is not balance.
>> > >
>> > > I have a idea about the balance policy:
>> > > (1) Firstly balance for each table
>> > > (2) Then, overall balance.
>> > >
>> > >
>> > > Schubert
>> > >
>> >
>>
>
>
>
> --
> Best Regards
> Anty Rao
>
-
Re: HBase balancer policy issueSchubert Zhang 2011-06-01, 04:13
We have encounter following issue:
1. A region have many files, the compacting takes long time.
2. But the balancer (default 5 minutes) close and move this region to
another server.
3. Then, the compacting start again.
4. Then, then balancer close and move it to another server.
Thus, the compacting cannot complete.
Now, we set the balancer interval to 30 minutes to remission this issue.
Schubert
On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
> If compacting, don't we interrupt it so we can close and move the
> region. You are worried about the compacting work done so far -- you
> don't want to lose it? So you are suggesting that a region should be
> able to say "No, not now! I'm busy?" (We'd need to distingush between
> a balancer 'move' and a move or close for any other region).
>
> St.Ack
>
> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> > When doing balance, Can we take into account the compaction status of
> > regions.
> > Currently, even the region is doing compaction, it can also be
> interrupted
> > to response to reassign.
> >
> >
> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
> wrote:
> >
> >> Thanks, I think it is HBASE-3373
> >>
> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>
> >> > I think your request is described in
> >> > https://issues.apache.org/jira/browse/HBASE-3373
> >> >
> >> > Practically speaking, the scenario below can hardly occur (in trunk,
> at
> >> > least).
> >> > If the tables are created with pre-split regions, the regions would be
> >> > round-robin distributed.
> >> > If the tables are created with single region, subsequent write
> operations
> >> > would cause the region split. Balancer would offload young regions to
> >> other
> >> > servers - see HBASE-3609
> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
> 0.90
> >> > branch.
> >> >
> >> > Refer to
> >> >
> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> >> > details.
> >> >
> >> > Cheers
> >> >
> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
> >> > wrote:
> >> >
> >> > > I have a question about HBase balancer.
> >> > >
> >> > > In release 0.90.x, it seems the balancer only regards the number of
> >> > regions
> >> > > and balance these regions into every regionserver.
> >> > >
> >> > > If we have two tables (A and B) now, each have 100 regions.
> >> > > Then, a extreme situation is:
> >> > >
> >> > > RegionsServer1: 100 regions, which all belong to table A
> >> > > RegionsServer2: 100 regions, which all belong to table B
> >> > >
> >> > > If my application access table B heavy, the almost all opetations
> hit
> >> > > RegionsServer2, it is not balance.
> >> > >
> >> > > I have a idea about the balance policy:
> >> > > (1) Firstly balance for each table
> >> > > (2) Then, overall balance.
> >> > >
> >> > >
> >> > > Schubert
> >> > >
> >> >
> >>
> >
> >
> >
> > --
> > Best Regards
> > Anty Rao
> >
>
1. A region have many files, the compacting takes long time.
2. But the balancer (default 5 minutes) close and move this region to
another server.
3. Then, the compacting start again.
4. Then, then balancer close and move it to another server.
Thus, the compacting cannot complete.
Now, we set the balancer interval to 30 minutes to remission this issue.
Schubert
On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
> If compacting, don't we interrupt it so we can close and move the
> region. You are worried about the compacting work done so far -- you
> don't want to lose it? So you are suggesting that a region should be
> able to say "No, not now! I'm busy?" (We'd need to distingush between
> a balancer 'move' and a move or close for any other region).
>
> St.Ack
>
> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> > When doing balance, Can we take into account the compaction status of
> > regions.
> > Currently, even the region is doing compaction, it can also be
> interrupted
> > to response to reassign.
> >
> >
> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
> wrote:
> >
> >> Thanks, I think it is HBASE-3373
> >>
> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>
> >> > I think your request is described in
> >> > https://issues.apache.org/jira/browse/HBASE-3373
> >> >
> >> > Practically speaking, the scenario below can hardly occur (in trunk,
> at
> >> > least).
> >> > If the tables are created with pre-split regions, the regions would be
> >> > round-robin distributed.
> >> > If the tables are created with single region, subsequent write
> operations
> >> > would cause the region split. Balancer would offload young regions to
> >> other
> >> > servers - see HBASE-3609
> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
> 0.90
> >> > branch.
> >> >
> >> > Refer to
> >> >
> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> >> > details.
> >> >
> >> > Cheers
> >> >
> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
> >> > wrote:
> >> >
> >> > > I have a question about HBase balancer.
> >> > >
> >> > > In release 0.90.x, it seems the balancer only regards the number of
> >> > regions
> >> > > and balance these regions into every regionserver.
> >> > >
> >> > > If we have two tables (A and B) now, each have 100 regions.
> >> > > Then, a extreme situation is:
> >> > >
> >> > > RegionsServer1: 100 regions, which all belong to table A
> >> > > RegionsServer2: 100 regions, which all belong to table B
> >> > >
> >> > > If my application access table B heavy, the almost all opetations
> hit
> >> > > RegionsServer2, it is not balance.
> >> > >
> >> > > I have a idea about the balance policy:
> >> > > (1) Firstly balance for each table
> >> > > (2) Then, overall balance.
> >> > >
> >> > >
> >> > > Schubert
> >> > >
> >> >
> >>
> >
> >
> >
> > --
> > Best Regards
> > Anty Rao
> >
>
-
Re: HBase balancer policy issueAnty 2011-06-01, 06:26
On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
> If compacting, don't we interrupt it so we can close and move the
> region. You are worried about the compacting work done so far -- you
> don't want to lose it? So you are suggesting that a region should be
> able to say "No, not now! I'm busy?" (We'd need to distingush between
> a balancer 'move' and a move or close for any other region).
>
Yes, that's exactly what i want to convey.
Can this consideration be included in balancer?
>
> St.Ack
>
> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> > When doing balance, Can we take into account the compaction status of
> > regions.
> > Currently, even the region is doing compaction, it can also be
> interrupted
> > to response to reassign.
> >
> >
> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
> wrote:
> >
> >> Thanks, I think it is HBASE-3373
> >>
> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>
> >> > I think your request is described in
> >> > https://issues.apache.org/jira/browse/HBASE-3373
> >> >
> >> > Practically speaking, the scenario below can hardly occur (in trunk,
> at
> >> > least).
> >> > If the tables are created with pre-split regions, the regions would be
> >> > round-robin distributed.
> >> > If the tables are created with single region, subsequent write
> operations
> >> > would cause the region split. Balancer would offload young regions to
> >> other
> >> > servers - see HBASE-3609
> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
> 0.90
> >> > branch.
> >> >
> >> > Refer to
> >> >
> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> >> > details.
> >> >
> >> > Cheers
> >> >
> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
> >> > wrote:
> >> >
> >> > > I have a question about HBase balancer.
> >> > >
> >> > > In release 0.90.x, it seems the balancer only regards the number of
> >> > regions
> >> > > and balance these regions into every regionserver.
> >> > >
> >> > > If we have two tables (A and B) now, each have 100 regions.
> >> > > Then, a extreme situation is:
> >> > >
> >> > > RegionsServer1: 100 regions, which all belong to table A
> >> > > RegionsServer2: 100 regions, which all belong to table B
> >> > >
> >> > > If my application access table B heavy, the almost all opetations
> hit
> >> > > RegionsServer2, it is not balance.
> >> > >
> >> > > I have a idea about the balance policy:
> >> > > (1) Firstly balance for each table
> >> > > (2) Then, overall balance.
> >> > >
> >> > >
> >> > > Schubert
> >> > >
> >> >
> >>
> >
> >
> >
> > --
> > Best Regards
> > Anty Rao
> >
>
--
Best Regards
Anty Rao
> If compacting, don't we interrupt it so we can close and move the
> region. You are worried about the compacting work done so far -- you
> don't want to lose it? So you are suggesting that a region should be
> able to say "No, not now! I'm busy?" (We'd need to distingush between
> a balancer 'move' and a move or close for any other region).
>
Yes, that's exactly what i want to convey.
Can this consideration be included in balancer?
>
> St.Ack
>
> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> > When doing balance, Can we take into account the compaction status of
> > regions.
> > Currently, even the region is doing compaction, it can also be
> interrupted
> > to response to reassign.
> >
> >
> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
> wrote:
> >
> >> Thanks, I think it is HBASE-3373
> >>
> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> >>
> >> > I think your request is described in
> >> > https://issues.apache.org/jira/browse/HBASE-3373
> >> >
> >> > Practically speaking, the scenario below can hardly occur (in trunk,
> at
> >> > least).
> >> > If the tables are created with pre-split regions, the regions would be
> >> > round-robin distributed.
> >> > If the tables are created with single region, subsequent write
> operations
> >> > would cause the region split. Balancer would offload young regions to
> >> other
> >> > servers - see HBASE-3609
> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
> 0.90
> >> > branch.
> >> >
> >> > Refer to
> >> >
> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> >> > details.
> >> >
> >> > Cheers
> >> >
> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
> >> > wrote:
> >> >
> >> > > I have a question about HBase balancer.
> >> > >
> >> > > In release 0.90.x, it seems the balancer only regards the number of
> >> > regions
> >> > > and balance these regions into every regionserver.
> >> > >
> >> > > If we have two tables (A and B) now, each have 100 regions.
> >> > > Then, a extreme situation is:
> >> > >
> >> > > RegionsServer1: 100 regions, which all belong to table A
> >> > > RegionsServer2: 100 regions, which all belong to table B
> >> > >
> >> > > If my application access table B heavy, the almost all opetations
> hit
> >> > > RegionsServer2, it is not balance.
> >> > >
> >> > > I have a idea about the balance policy:
> >> > > (1) Firstly balance for each table
> >> > > (2) Then, overall balance.
> >> > >
> >> > >
> >> > > Schubert
> >> > >
> >> >
> >>
> >
> >
> >
> > --
> > Best Regards
> > Anty Rao
> >
>
--
Best Regards
Anty Rao
-
Re: HBase balancer policy issueTed Yu 2011-06-01, 16:32
I logged https://issues.apache.org/jira/browse/HBASE-3943
This seems to be a popular request.
I will wait for other developers' comment first.
On Tue, May 31, 2011 at 11:26 PM, Anty <[EMAIL PROTECTED]> wrote:
> On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > If compacting, don't we interrupt it so we can close and move the
> > region. You are worried about the compacting work done so far -- you
> > don't want to lose it? So you are suggesting that a region should be
> > able to say "No, not now! I'm busy?" (We'd need to distingush between
> > a balancer 'move' and a move or close for any other region).
> >
> Yes, that's exactly what i want to convey.
> Can this consideration be included in balancer?
>
> >
> > St.Ack
> >
> > On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> > > When doing balance, Can we take into account the compaction status of
> > > regions.
> > > Currently, even the region is doing compaction, it can also be
> > interrupted
> > > to response to reassign.
> > >
> > >
> > > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
> > wrote:
> > >
> > >> Thanks, I think it is HBASE-3373
> > >>
> > >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >>
> > >> > I think your request is described in
> > >> > https://issues.apache.org/jira/browse/HBASE-3373
> > >> >
> > >> > Practically speaking, the scenario below can hardly occur (in trunk,
> > at
> > >> > least).
> > >> > If the tables are created with pre-split regions, the regions would
> be
> > >> > round-robin distributed.
> > >> > If the tables are created with single region, subsequent write
> > operations
> > >> > would cause the region split. Balancer would offload young regions
> to
> > >> other
> > >> > servers - see HBASE-3609
> > >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
> > 0.90
> > >> > branch.
> > >> >
> > >> > Refer to
> > >> >
> > http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> > >> > details.
> > >> >
> > >> > Cheers
> > >> >
> > >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]
> >
> > >> > wrote:
> > >> >
> > >> > > I have a question about HBase balancer.
> > >> > >
> > >> > > In release 0.90.x, it seems the balancer only regards the number
> of
> > >> > regions
> > >> > > and balance these regions into every regionserver.
> > >> > >
> > >> > > If we have two tables (A and B) now, each have 100 regions.
> > >> > > Then, a extreme situation is:
> > >> > >
> > >> > > RegionsServer1: 100 regions, which all belong to table A
> > >> > > RegionsServer2: 100 regions, which all belong to table B
> > >> > >
> > >> > > If my application access table B heavy, the almost all opetations
> > hit
> > >> > > RegionsServer2, it is not balance.
> > >> > >
> > >> > > I have a idea about the balance policy:
> > >> > > (1) Firstly balance for each table
> > >> > > (2) Then, overall balance.
> > >> > >
> > >> > >
> > >> > > Schubert
> > >> > >
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Best Regards
> > > Anty Rao
> > >
> >
>
>
>
> --
> Best Regards
> Anty Rao
>
This seems to be a popular request.
I will wait for other developers' comment first.
On Tue, May 31, 2011 at 11:26 PM, Anty <[EMAIL PROTECTED]> wrote:
> On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > If compacting, don't we interrupt it so we can close and move the
> > region. You are worried about the compacting work done so far -- you
> > don't want to lose it? So you are suggesting that a region should be
> > able to say "No, not now! I'm busy?" (We'd need to distingush between
> > a balancer 'move' and a move or close for any other region).
> >
> Yes, that's exactly what i want to convey.
> Can this consideration be included in balancer?
>
> >
> > St.Ack
> >
> > On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> > > When doing balance, Can we take into account the compaction status of
> > > regions.
> > > Currently, even the region is doing compaction, it can also be
> > interrupted
> > > to response to reassign.
> > >
> > >
> > > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
> > wrote:
> > >
> > >> Thanks, I think it is HBASE-3373
> > >>
> > >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
> > >>
> > >> > I think your request is described in
> > >> > https://issues.apache.org/jira/browse/HBASE-3373
> > >> >
> > >> > Practically speaking, the scenario below can hardly occur (in trunk,
> > at
> > >> > least).
> > >> > If the tables are created with pre-split regions, the regions would
> be
> > >> > round-robin distributed.
> > >> > If the tables are created with single region, subsequent write
> > operations
> > >> > would cause the region split. Balancer would offload young regions
> to
> > >> other
> > >> > servers - see HBASE-3609
> > >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
> > 0.90
> > >> > branch.
> > >> >
> > >> > Refer to
> > >> >
> > http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> > >> > details.
> > >> >
> > >> > Cheers
> > >> >
> > >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]
> >
> > >> > wrote:
> > >> >
> > >> > > I have a question about HBase balancer.
> > >> > >
> > >> > > In release 0.90.x, it seems the balancer only regards the number
> of
> > >> > regions
> > >> > > and balance these regions into every regionserver.
> > >> > >
> > >> > > If we have two tables (A and B) now, each have 100 regions.
> > >> > > Then, a extreme situation is:
> > >> > >
> > >> > > RegionsServer1: 100 regions, which all belong to table A
> > >> > > RegionsServer2: 100 regions, which all belong to table B
> > >> > >
> > >> > > If my application access table B heavy, the almost all opetations
> > hit
> > >> > > RegionsServer2, it is not balance.
> > >> > >
> > >> > > I have a idea about the balance policy:
> > >> > > (1) Firstly balance for each table
> > >> > > (2) Then, overall balance.
> > >> > >
> > >> > >
> > >> > > Schubert
> > >> > >
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Best Regards
> > > Anty Rao
> > >
> >
>
>
>
> --
> Best Regards
> Anty Rao
>
-
Re: HBase balancer policy issueStack 2011-06-01, 17:25
Interesting. I suppose if the balancer remembered what it had
recently moved and let some time elapse before it moved the recently
moved, would that do?
St.Ack
On Tue, May 31, 2011 at 11:26 PM, Anty <[EMAIL PROTECTED]> wrote:
> On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
>
>> If compacting, don't we interrupt it so we can close and move the
>> region. You are worried about the compacting work done so far -- you
>> don't want to lose it? So you are suggesting that a region should be
>> able to say "No, not now! I'm busy?" (We'd need to distingush between
>> a balancer 'move' and a move or close for any other region).
>>
> Yes, that's exactly what i want to convey.
> Can this consideration be included in balancer?
>
>>
>> St.Ack
>>
>> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
>> > When doing balance, Can we take into account the compaction status of
>> > regions.
>> > Currently, even the region is doing compaction, it can also be
>> interrupted
>> > to response to reassign.
>> >
>> >
>> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> Thanks, I think it is HBASE-3373
>> >>
>> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> >>
>> >> > I think your request is described in
>> >> > https://issues.apache.org/jira/browse/HBASE-3373
>> >> >
>> >> > Practically speaking, the scenario below can hardly occur (in trunk,
>> at
>> >> > least).
>> >> > If the tables are created with pre-split regions, the regions would be
>> >> > round-robin distributed.
>> >> > If the tables are created with single region, subsequent write
>> operations
>> >> > would cause the region split. Balancer would offload young regions to
>> >> other
>> >> > servers - see HBASE-3609
>> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
>> 0.90
>> >> > branch.
>> >> >
>> >> > Refer to
>> >> >
>> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
>> >> > details.
>> >> >
>> >> > Cheers
>> >> >
>> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
>> >> > wrote:
>> >> >
>> >> > > I have a question about HBase balancer.
>> >> > >
>> >> > > In release 0.90.x, it seems the balancer only regards the number of
>> >> > regions
>> >> > > and balance these regions into every regionserver.
>> >> > >
>> >> > > If we have two tables (A and B) now, each have 100 regions.
>> >> > > Then, a extreme situation is:
>> >> > >
>> >> > > RegionsServer1: 100 regions, which all belong to table A
>> >> > > RegionsServer2: 100 regions, which all belong to table B
>> >> > >
>> >> > > If my application access table B heavy, the almost all opetations
>> hit
>> >> > > RegionsServer2, it is not balance.
>> >> > >
>> >> > > I have a idea about the balance policy:
>> >> > > (1) Firstly balance for each table
>> >> > > (2) Then, overall balance.
>> >> > >
>> >> > >
>> >> > > Schubert
>> >> > >
>> >> >
>> >>
>> >
>> >
>> >
>> > --
>> > Best Regards
>> > Anty Rao
>> >
>>
>
>
>
> --
> Best Regards
> Anty Rao
>
recently moved and let some time elapse before it moved the recently
moved, would that do?
St.Ack
On Tue, May 31, 2011 at 11:26 PM, Anty <[EMAIL PROTECTED]> wrote:
> On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
>
>> If compacting, don't we interrupt it so we can close and move the
>> region. You are worried about the compacting work done so far -- you
>> don't want to lose it? So you are suggesting that a region should be
>> able to say "No, not now! I'm busy?" (We'd need to distingush between
>> a balancer 'move' and a move or close for any other region).
>>
> Yes, that's exactly what i want to convey.
> Can this consideration be included in balancer?
>
>>
>> St.Ack
>>
>> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
>> > When doing balance, Can we take into account the compaction status of
>> > regions.
>> > Currently, even the region is doing compaction, it can also be
>> interrupted
>> > to response to reassign.
>> >
>> >
>> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
>> wrote:
>> >
>> >> Thanks, I think it is HBASE-3373
>> >>
>> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>> >>
>> >> > I think your request is described in
>> >> > https://issues.apache.org/jira/browse/HBASE-3373
>> >> >
>> >> > Practically speaking, the scenario below can hardly occur (in trunk,
>> at
>> >> > least).
>> >> > If the tables are created with pre-split regions, the regions would be
>> >> > round-robin distributed.
>> >> > If the tables are created with single region, subsequent write
>> operations
>> >> > would cause the region split. Balancer would offload young regions to
>> >> other
>> >> > servers - see HBASE-3609
>> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
>> 0.90
>> >> > branch.
>> >> >
>> >> > Refer to
>> >> >
>> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
>> >> > details.
>> >> >
>> >> > Cheers
>> >> >
>> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <[EMAIL PROTECTED]>
>> >> > wrote:
>> >> >
>> >> > > I have a question about HBase balancer.
>> >> > >
>> >> > > In release 0.90.x, it seems the balancer only regards the number of
>> >> > regions
>> >> > > and balance these regions into every regionserver.
>> >> > >
>> >> > > If we have two tables (A and B) now, each have 100 regions.
>> >> > > Then, a extreme situation is:
>> >> > >
>> >> > > RegionsServer1: 100 regions, which all belong to table A
>> >> > > RegionsServer2: 100 regions, which all belong to table B
>> >> > >
>> >> > > If my application access table B heavy, the almost all opetations
>> hit
>> >> > > RegionsServer2, it is not balance.
>> >> > >
>> >> > > I have a idea about the balance policy:
>> >> > > (1) Firstly balance for each table
>> >> > > (2) Then, overall balance.
>> >> > >
>> >> > >
>> >> > > Schubert
>> >> > >
>> >> >
>> >>
>> >
>> >
>> >
>> > --
>> > Best Regards
>> > Anty Rao
>> >
>>
>
>
>
> --
> Best Regards
> Anty Rao
>
-
Re: HBase balancer policy issueAnty 2011-06-02, 08:19
if region A is doing compaction , load balancer ask region server to close
region A
if set hbase.hstore.close.check.interval = 0,
the region A can't be closed until the completion of compaction,
during this period, the region A can also accept mutation request?
Am I right?
Does this have bad affect on load balancer?
On Thu, Jun 2, 2011 at 1:25 AM, Stack <[EMAIL PROTECTED]> wrote:
> Interesting. I suppose if the balancer remembered what it had
> recently moved and let some time elapse before it moved the recently
> moved, would that do?
> St.Ack
>
> On Tue, May 31, 2011 at 11:26 PM, Anty <[EMAIL PROTECTED]> wrote:
> > On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
> >
> >> If compacting, don't we interrupt it so we can close and move the
> >> region. You are worried about the compacting work done so far -- you
> >> don't want to lose it? So you are suggesting that a region should be
> >> able to say "No, not now! I'm busy?" (We'd need to distingush between
> >> a balancer 'move' and a move or close for any other region).
> >>
> > Yes, that's exactly what i want to convey.
> > Can this consideration be included in balancer?
> >
> >>
> >> St.Ack
> >>
> >> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> >> > When doing balance, Can we take into account the compaction status of
> >> > regions.
> >> > Currently, even the region is doing compaction, it can also be
> >> interrupted
> >> > to response to reassign.
> >> >
> >> >
> >> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
> >> wrote:
> >> >
> >> >> Thanks, I think it is HBASE-3373
> >> >>
> >> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]>
> wrote:
> >> >>
> >> >> > I think your request is described in
> >> >> > https://issues.apache.org/jira/browse/HBASE-3373
> >> >> >
> >> >> > Practically speaking, the scenario below can hardly occur (in
> trunk,
> >> at
> >> >> > least).
> >> >> > If the tables are created with pre-split regions, the regions would
> be
> >> >> > round-robin distributed.
> >> >> > If the tables are created with single region, subsequent write
> >> operations
> >> >> > would cause the region split. Balancer would offload young regions
> to
> >> >> other
> >> >> > servers - see HBASE-3609
> >> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
> >> 0.90
> >> >> > branch.
> >> >> >
> >> >> > Refer to
> >> >> >
> >>
> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> >> >> > details.
> >> >> >
> >> >> > Cheers
> >> >> >
> >> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <
> [EMAIL PROTECTED]>
> >> >> > wrote:
> >> >> >
> >> >> > > I have a question about HBase balancer.
> >> >> > >
> >> >> > > In release 0.90.x, it seems the balancer only regards the number
> of
> >> >> > regions
> >> >> > > and balance these regions into every regionserver.
> >> >> > >
> >> >> > > If we have two tables (A and B) now, each have 100 regions.
> >> >> > > Then, a extreme situation is:
> >> >> > >
> >> >> > > RegionsServer1: 100 regions, which all belong to table A
> >> >> > > RegionsServer2: 100 regions, which all belong to table B
> >> >> > >
> >> >> > > If my application access table B heavy, the almost all opetations
> >> hit
> >> >> > > RegionsServer2, it is not balance.
> >> >> > >
> >> >> > > I have a idea about the balance policy:
> >> >> > > (1) Firstly balance for each table
> >> >> > > (2) Then, overall balance.
> >> >> > >
> >> >> > >
> >> >> > > Schubert
> >> >> > >
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards
> >> > Anty Rao
> >> >
> >>
> >
> >
> >
> > --
> > Best Regards
> > Anty Rao
> >
>
--
Best Regards
Anty Rao
region A
if set hbase.hstore.close.check.interval = 0,
the region A can't be closed until the completion of compaction,
during this period, the region A can also accept mutation request?
Am I right?
Does this have bad affect on load balancer?
On Thu, Jun 2, 2011 at 1:25 AM, Stack <[EMAIL PROTECTED]> wrote:
> Interesting. I suppose if the balancer remembered what it had
> recently moved and let some time elapse before it moved the recently
> moved, would that do?
> St.Ack
>
> On Tue, May 31, 2011 at 11:26 PM, Anty <[EMAIL PROTECTED]> wrote:
> > On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
> >
> >> If compacting, don't we interrupt it so we can close and move the
> >> region. You are worried about the compacting work done so far -- you
> >> don't want to lose it? So you are suggesting that a region should be
> >> able to say "No, not now! I'm busy?" (We'd need to distingush between
> >> a balancer 'move' and a move or close for any other region).
> >>
> > Yes, that's exactly what i want to convey.
> > Can this consideration be included in balancer?
> >
> >>
> >> St.Ack
> >>
> >> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> >> > When doing balance, Can we take into account the compaction status of
> >> > regions.
> >> > Currently, even the region is doing compaction, it can also be
> >> interrupted
> >> > to response to reassign.
> >> >
> >> >
> >> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]>
> >> wrote:
> >> >
> >> >> Thanks, I think it is HBASE-3373
> >> >>
> >> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]>
> wrote:
> >> >>
> >> >> > I think your request is described in
> >> >> > https://issues.apache.org/jira/browse/HBASE-3373
> >> >> >
> >> >> > Practically speaking, the scenario below can hardly occur (in
> trunk,
> >> at
> >> >> > least).
> >> >> > If the tables are created with pre-split regions, the regions would
> be
> >> >> > round-robin distributed.
> >> >> > If the tables are created with single region, subsequent write
> >> operations
> >> >> > would cause the region split. Balancer would offload young regions
> to
> >> >> other
> >> >> > servers - see HBASE-3609
> >> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not in
> >> 0.90
> >> >> > branch.
> >> >> >
> >> >> > Refer to
> >> >> >
> >>
> http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> >> >> > details.
> >> >> >
> >> >> > Cheers
> >> >> >
> >> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <
> [EMAIL PROTECTED]>
> >> >> > wrote:
> >> >> >
> >> >> > > I have a question about HBase balancer.
> >> >> > >
> >> >> > > In release 0.90.x, it seems the balancer only regards the number
> of
> >> >> > regions
> >> >> > > and balance these regions into every regionserver.
> >> >> > >
> >> >> > > If we have two tables (A and B) now, each have 100 regions.
> >> >> > > Then, a extreme situation is:
> >> >> > >
> >> >> > > RegionsServer1: 100 regions, which all belong to table A
> >> >> > > RegionsServer2: 100 regions, which all belong to table B
> >> >> > >
> >> >> > > If my application access table B heavy, the almost all opetations
> >> hit
> >> >> > > RegionsServer2, it is not balance.
> >> >> > >
> >> >> > > I have a idea about the balance policy:
> >> >> > > (1) Firstly balance for each table
> >> >> > > (2) Then, overall balance.
> >> >> > >
> >> >> > >
> >> >> > > Schubert
> >> >> > >
> >> >> >
> >> >>
> >> >
> >> >
> >> >
> >> > --
> >> > Best Regards
> >> > Anty Rao
> >> >
> >>
> >
> >
> >
> > --
> > Best Regards
> > Anty Rao
> >
>
--
Best Regards
Anty Rao
-
Re: HBase balancer policy issueSchubert Zhang 2011-06-02, 15:41
Thanks Ted.
On Thu, Jun 2, 2011 at 4:19 PM, Anty <[EMAIL PROTECTED]> wrote:
> if region A is doing compaction , load balancer ask region server to close
> region A
> if set hbase.hstore.close.check.interval = 0,
> the region A can't be closed until the completion of compaction,
> during this period, the region A can also accept mutation request?
> Am I right?
> Does this have bad affect on load balancer?
>
> On Thu, Jun 2, 2011 at 1:25 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > Interesting. I suppose if the balancer remembered what it had
> > recently moved and let some time elapse before it moved the recently
> > moved, would that do?
> > St.Ack
> >
> > On Tue, May 31, 2011 at 11:26 PM, Anty <[EMAIL PROTECTED]> wrote:
> > > On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
> > >
> > >> If compacting, don't we interrupt it so we can close and move the
> > >> region. You are worried about the compacting work done so far -- you
> > >> don't want to lose it? So you are suggesting that a region should be
> > >> able to say "No, not now! I'm busy?" (We'd need to distingush between
> > >> a balancer 'move' and a move or close for any other region).
> > >>
> > > Yes, that's exactly what i want to convey.
> > > Can this consideration be included in balancer?
> > >
> > >>
> > >> St.Ack
> > >>
> > >> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> > >> > When doing balance, Can we take into account the compaction status
> of
> > >> > regions.
> > >> > Currently, even the region is doing compaction, it can also be
> > >> interrupted
> > >> > to response to reassign.
> > >> >
> > >> >
> > >> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]
> >
> > >> wrote:
> > >> >
> > >> >> Thanks, I think it is HBASE-3373
> > >> >>
> > >> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]>
> > wrote:
> > >> >>
> > >> >> > I think your request is described in
> > >> >> > https://issues.apache.org/jira/browse/HBASE-3373
> > >> >> >
> > >> >> > Practically speaking, the scenario below can hardly occur (in
> > trunk,
> > >> at
> > >> >> > least).
> > >> >> > If the tables are created with pre-split regions, the regions
> would
> > be
> > >> >> > round-robin distributed.
> > >> >> > If the tables are created with single region, subsequent write
> > >> operations
> > >> >> > would cause the region split. Balancer would offload young
> regions
> > to
> > >> >> other
> > >> >> > servers - see HBASE-3609
> > >> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not
> in
> > >> 0.90
> > >> >> > branch.
> > >> >> >
> > >> >> > Refer to
> > >> >> >
> > >>
> > http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> > >> >> > details.
> > >> >> >
> > >> >> > Cheers
> > >> >> >
> > >> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <
> > [EMAIL PROTECTED]>
> > >> >> > wrote:
> > >> >> >
> > >> >> > > I have a question about HBase balancer.
> > >> >> > >
> > >> >> > > In release 0.90.x, it seems the balancer only regards the
> number
> > of
> > >> >> > regions
> > >> >> > > and balance these regions into every regionserver.
> > >> >> > >
> > >> >> > > If we have two tables (A and B) now, each have 100 regions.
> > >> >> > > Then, a extreme situation is:
> > >> >> > >
> > >> >> > > RegionsServer1: 100 regions, which all belong to table A
> > >> >> > > RegionsServer2: 100 regions, which all belong to table B
> > >> >> > >
> > >> >> > > If my application access table B heavy, the almost all
> opetations
> > >> hit
> > >> >> > > RegionsServer2, it is not balance.
> > >> >> > >
> > >> >> > > I have a idea about the balance policy:
> > >> >> > > (1) Firstly balance for each table
> > >> >> > > (2) Then, overall balance.
> > >> >> > >
> > >> >> > >
> > >> >> > > Schubert
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Best Regards
> > >> > Anty Rao
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Best Regards
> > > Anty Rao
On Thu, Jun 2, 2011 at 4:19 PM, Anty <[EMAIL PROTECTED]> wrote:
> if region A is doing compaction , load balancer ask region server to close
> region A
> if set hbase.hstore.close.check.interval = 0,
> the region A can't be closed until the completion of compaction,
> during this period, the region A can also accept mutation request?
> Am I right?
> Does this have bad affect on load balancer?
>
> On Thu, Jun 2, 2011 at 1:25 AM, Stack <[EMAIL PROTECTED]> wrote:
>
> > Interesting. I suppose if the balancer remembered what it had
> > recently moved and let some time elapse before it moved the recently
> > moved, would that do?
> > St.Ack
> >
> > On Tue, May 31, 2011 at 11:26 PM, Anty <[EMAIL PROTECTED]> wrote:
> > > On Wed, Jun 1, 2011 at 11:02 AM, Stack <[EMAIL PROTECTED]> wrote:
> > >
> > >> If compacting, don't we interrupt it so we can close and move the
> > >> region. You are worried about the compacting work done so far -- you
> > >> don't want to lose it? So you are suggesting that a region should be
> > >> able to say "No, not now! I'm busy?" (We'd need to distingush between
> > >> a balancer 'move' and a move or close for any other region).
> > >>
> > > Yes, that's exactly what i want to convey.
> > > Can this consideration be included in balancer?
> > >
> > >>
> > >> St.Ack
> > >>
> > >> On Tue, May 31, 2011 at 7:44 PM, Anty <[EMAIL PROTECTED]> wrote:
> > >> > When doing balance, Can we take into account the compaction status
> of
> > >> > regions.
> > >> > Currently, even the region is doing compaction, it can also be
> > >> interrupted
> > >> > to response to reassign.
> > >> >
> > >> >
> > >> > On Sun, May 29, 2011 at 12:57 AM, Schubert Zhang <[EMAIL PROTECTED]
> >
> > >> wrote:
> > >> >
> > >> >> Thanks, I think it is HBASE-3373
> > >> >>
> > >> >> On Fri, May 20, 2011 at 11:33 PM, Ted Yu <[EMAIL PROTECTED]>
> > wrote:
> > >> >>
> > >> >> > I think your request is described in
> > >> >> > https://issues.apache.org/jira/browse/HBASE-3373
> > >> >> >
> > >> >> > Practically speaking, the scenario below can hardly occur (in
> > trunk,
> > >> at
> > >> >> > least).
> > >> >> > If the tables are created with pre-split regions, the regions
> would
> > be
> > >> >> > round-robin distributed.
> > >> >> > If the tables are created with single region, subsequent write
> > >> operations
> > >> >> > would cause the region split. Balancer would offload young
> regions
> > to
> > >> >> other
> > >> >> > servers - see HBASE-3609
> > >> >> > <https://issues.apache.org/jira/browse/HBASE-3609>which is not
> in
> > >> 0.90
> > >> >> > branch.
> > >> >> >
> > >> >> > Refer to
> > >> >> >
> > >>
> > http://zhihongyu.blogspot.com/2011/04/load-balancer-in-hbase-090.htmlfor
> > >> >> > details.
> > >> >> >
> > >> >> > Cheers
> > >> >> >
> > >> >> > On Fri, May 20, 2011 at 12:25 AM, Schubert Zhang <
> > [EMAIL PROTECTED]>
> > >> >> > wrote:
> > >> >> >
> > >> >> > > I have a question about HBase balancer.
> > >> >> > >
> > >> >> > > In release 0.90.x, it seems the balancer only regards the
> number
> > of
> > >> >> > regions
> > >> >> > > and balance these regions into every regionserver.
> > >> >> > >
> > >> >> > > If we have two tables (A and B) now, each have 100 regions.
> > >> >> > > Then, a extreme situation is:
> > >> >> > >
> > >> >> > > RegionsServer1: 100 regions, which all belong to table A
> > >> >> > > RegionsServer2: 100 regions, which all belong to table B
> > >> >> > >
> > >> >> > > If my application access table B heavy, the almost all
> opetations
> > >> hit
> > >> >> > > RegionsServer2, it is not balance.
> > >> >> > >
> > >> >> > > I have a idea about the balance policy:
> > >> >> > > (1) Firstly balance for each table
> > >> >> > > (2) Then, overall balance.
> > >> >> > >
> > >> >> > >
> > >> >> > > Schubert
> > >> >> > >
> > >> >> >
> > >> >>
> > >> >
> > >> >
> > >> >
> > >> > --
> > >> > Best Regards
> > >> > Anty Rao
> > >> >
> > >>
> > >
> > >
> > >
> > > --
> > > Best Regards
> > > Anty Rao
