Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - hbase-0.94.6.1 balancer issue


Copy link to this message
-
Re: hbase-0.94.6.1 balancer issue
Samir Ahmic 2013-04-12, 18:06
Hi, J-D

Well at this moment i have that edge case with only one region per table:).
Like i said i was using 0.90 for  long time and regions were distributed
evenly  on all RSs regardless on region per table ratio. Here is
what confused me (like i said i have 2 nodes cluster distributed mode):

start-hbase ------>  tables(regions) are distributed evenlyon two RSs (As
expected)
stop one RS -------> all tables(regions) are moved to remaining RS (as
expected)
start RS that was down ---> run balancer ---> LOG:2013-04-12 19:47:20,725
INFO org.apache.hadoop.hbase.master.LoadBalancer: Skipping load balancing
because balanced cluster; servers=2 regions=1 average=0.5 mostloaded=1
leastloaded=0 ----> all tables(regions) stayed  on one server (this is what
i did not expect ?????) :)

Here is is part of status 'detailed' from shell ater i start RS that was
down and run balancer:

hbase(main):001:0> status 'detailed'
version 0.94.6.1
0 regionsInTransition
master coprocessors: []
2 live servers
    172.17.33.2:60020 1365787755294
        requestsPerSecond=0, numberOfOnlineRegions=0, usedHeapMB=38,
maxHeapMB=3487
    172.17.33.3:60020 1365777858778
        requestsPerSecond=0, numberOfOnlineRegions=49, usedHeapMB=53,
maxHeapMB=3487

So because i have 1 regions per table regions were not rebalances after
start RS  that was down?

Thanks
Samir
On Fri, Apr 12, 2013 at 7:17 PM, Jean-Daniel Cryans <[EMAIL PROTECTED]>wrote:

> Samir,
>
> When you say "And at what point balancer will start redistribute regions to
> second server", do you mean that when you look at the master's web UI you
> see that one region server has 0 region? That would be a problem. Else,
> that line you posted in your original message should be repeated for each
> table, and globally the regions should all be correctly distributed...
> unless there's an edge case where when you have only tables with 1 region
> it puts them all on the same server :)
>
> Thx,
>
> J-D
>
>
> On Fri, Apr 12, 2013 at 12:37 PM, Samir Ahmic <[EMAIL PROTECTED]>
> wrote:
>
> > Thanks for explaining Jean-Marc,
> >
> > We are using 0.90.4 for very long time and balancing was based on total
> > number of regions.That is why i was surprised with balancer log on 0.94.
> > Well i'm more ops guy then dev i handle what other develop :)
> >
> > Regards
> >
> >
> > On Fri, Apr 12, 2013 at 6:24 PM, Jean-Marc Spaggiari <
> > [EMAIL PROTECTED]> wrote:
> >
> > > Hi Samir,
> > >
> > > Since regions are balanced per table, as soon as you will have more
> than
> > > one region in your table, balancer will start to balance the regions
> over
> > > the servers.
> > >
> > > You can split some of those tables and will you start to see HBase
> > balance
> > > them. This is normal behavior for 0.94. I don't know for versions
> before
> > > that.
> > >
> > > Also, are you sure you need 48 tables? And not less tables with more
> CFs?
> > >
> > > JM
> > >
> > > 2013/4/12 Samir Ahmic <[EMAIL PROTECTED]>
> > >
> > > > Hi, JM
> > > >
> > > > I have 48 tables and as you said it is 1 region per table since i did
> > not
> > > > reach splitting limit yet. So this is normal behavior  in 0.94.6.1
> > > version
> > > > ?  And at what point balancer will start redistribute regions to
> second
> > > > server ?
> > > >
> > > > Thanks
> > > > Samir
> > > >
> > > >
> > > > On Fri, Apr 12, 2013 at 6:06 PM, Jean-Marc Spaggiari <
> > > > [EMAIL PROTECTED]> wrote:
> > > >
> > > > > Hi Samir,
> > > > >
> > > > > Regions are balancer per table.
> > > > >
> > > > > So if you have 48 regions within the same table, it should be split
> > > about
> > > > > 24 on each server.
> > > > >
> > > > > But if you have 48 tables with 1 region each, the for each table,
> the
> > > > > balancer will see only 1 region and will display the message you
> saw.
> > > > >
> > > > > Have you looked at the UI? What do you have in it? Can you please
> > > confirm
> > > > > if yo uhave 48 tables or 1 table?
> > > > >
> > > > > Thanks,
> > > > >
> > > > > JM