Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Smarter Region assignment policy?


Copy link to this message
-
Re: Smarter Region assignment policy?
lars hofhansl 2013-07-23, 17:41
Is there ever a reason to not attempt retaining assignments?

----- Original Message -----
From: Ted Yu <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Cc:
Sent: Tuesday, July 23, 2013 8:58 AM
Subject: Re: Smarter Region assignment policy?

In trunk, we have the following code in HMaster#enableTable():

    this.executorService.submit(new EnableTableHandler(this, tableName,

      catalogTracker, assignmentManager, tableLockManager, false
).prepare());
The boolean value of false means the assignment wouldn't retain region
locations (compared to the locations prior to disabling).
In 0.94, I see similar code.

Looks like we can add new API where user can specify whether
retainAssignment should be used.

Cheers

On Mon, Jul 22, 2013 at 4:21 PM, Vladimir Rodionov
<[EMAIL PROTECTED]>wrote:

> Is not this https://issues.apache.org/jira/browse/HBASE-6143?
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: lars hofhansl [[EMAIL PROTECTED]]
> Sent: Monday, July 22, 2013 4:04 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Smarter Region assignment policy?
>
> Hmm... So that's an interesting bug then. Might filing a jira?
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Vladimir Rodionov <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; lars hofhansl <
> [EMAIL PROTECTED]>
> Cc:
> Sent: Monday, July 22, 2013 3:30 PM
> Subject: RE: Smarter Region assignment policy?
>
> OK, you right and I was not. If I do not disable/enables tables after
> cluster restart  all regions seem get assigned correctly.
> I think its disabling/enabling tables shuffles regions in a bad way.
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: lars hofhansl [[EMAIL PROTECTED]]
> Sent: Monday, July 22, 2013 12:13 PM
> To: [EMAIL PROTECTED]; lars hofhansl
> Subject: Re: Smarter Region assignment policy?
>
> This is the one:
> https://issues.apache.org/jira/browse/HBASE-4402
>
>
> I assume you let the master wait sufficiently to give all RegionServer a
> chance to sign in?
>
> -- Lars
> ________________________________
> From: lars hofhansl <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Monday, July 22, 2013 11:55 AM
> Subject: Re: Smarter Region assignment policy?
>
>
> I faintly remember there was a jira that attempted to assign the regions
> to the same region servers after a restart based on existing .META.
> information (if possible).
> Will try to find that and see why it is not working as expected.
>
>
> -- Lars
>
>
>
> ________________________________
> From: Vladimir Rodionov <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> Sent: Monday, July 22, 2013 11:43 AM
> Subject: RE: Smarter Region assignment policy?
>
>
> Yes, its related but not what I am looking for.
>
> Locality index is approaching to 100% after major compaction of all
> tables. Its often > 90% during regular operation of a cluster,
> but is far below 50% when cluster restarts. Can assignment manager take
> into account previous region assignments on
> node/cluster start up? That is what I am looking for. How hard is to
> implement this feature?
>
> Best regards,
> Vladimir Rodionov
> Principal Platform Engineer
> Carrier IQ, www.carrieriq.com
> e-mail: [EMAIL PROTECTED]
>
> ________________________________________
> From: Ted Yu [[EMAIL PROTECTED]]
> Sent: Monday, July 22, 2013 11:06 AM
> To: [EMAIL PROTECTED]
> Subject: Re: Smarter Region assignment policy?
>
> Please take a look at https://issues.apache.org/jira/browse/HBASE-4755
>
> On Mon, Jul 22, 2013 at 10:56 AM, Vladimir Rodionov <
> [EMAIL PROTECTED]
> > wrote:
>
> > It seems that current (0.94.6) region assignment on start up policy is
> not
> > that smart and does not utilize hdfs block locality