Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Pull instant schema updating out?


+
Jean-Daniel Cryans 2012-04-02, 23:56
+
Stack 2012-04-03, 00:03
+
Todd Lipcon 2012-04-02, 23:58
Copy link to this message
-
Re: Pull instant schema updating out?
"instant schema alter" should endure the scenarios Todd mentioned.

I think we just need to simplify the usage of MonitoredTask there.

On Apr 2, 2012, at 4:58 PM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> I agree having two implementations seems like a big mess. Any sense
> which one is closer to working for the average user (i.e without
> provisions like "don't split or crash servers mid-alter")?
>
> On Mon, Apr 2, 2012 at 4:56 PM, Jean-Daniel Cryans <[EMAIL PROTECTED]> wrote:
>> Hi nifty devs,
>>
>> After encountering HBASE-5702, I started playing with instant schema
>> updating (HBASE-4213) a bit more and I must say that it's a bit rough
>> which makes me wonder... should we pull that code out?
>>
>> We're in this "interesting" situation in 0.94 where we have two
>> different ways to alter tables without disabling them and I don't
>> trust either. I'm pretty sure most of the devs don't even know which
>> one takes precedence over the other when both are enabled without
>> looking at the code. Well, right now hbase.online.schema.update.enable
>> needs to be enabled in order to have
>> hbase.instant.schema.alter.enabled working. If only the former is
>> enabled the master handles the alter, else if both are enabled then
>> it's going to be done via ZK although the master still keeps track of
>> it.
>>
>> So the differences between both IIUC:
>>
>>  - "Online schema update" is a rolling close/open of all the regions
>> so that they pick up the new HTD. It's handled by the master and has
>> been in since 0.92. I've used it quite a bit when running other tests
>> and it's ok as long as regions are not splitting and RS are not
>> shutting down. We also enabled it on our clusters here since our
>> regions don't tend to move that much.
>>
>>  - "Instant schema alter" is instant in the sense that all the regions
>> are asked to close from the get-go but effectively the region servers
>> can only close one region at a time. The state of the alter is kept in
>> ZK and the master has a bunch of watches and logs the progress. It's
>> new since 0.94 and I'm not sure if anyone is using it. I've tested it
>> a bit and at the moment I can say the the MonitedTasks handling needs
>> to be redone, it logs way too much information in the log, but the few
>> alters I ran worked... It's just a bit hard to know when they're done.
>>
>> FWIW we could pull either or both out, but instant schema alter hasn't
>> been in a released version yet so it's unlikely it'll bother someone
>> while the other is already in use (like here).
>>
>> Opinions?
>>
>> Thanks,
>>
>> J-D
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
+
lars hofhansl 2012-04-03, 01:04
+
Stack 2012-04-03, 04:05
+
Stack 2012-04-03, 04:38
+
Ted Yu 2012-04-03, 04:11
+
lars hofhansl 2012-04-03, 06:07
+
yuzhihong@... 2012-04-03, 10:20
+
Stack 2012-04-03, 14:50
+
Andrew Purtell 2012-04-03, 17:19
+
lars hofhansl 2012-04-03, 16:14
+
Ted Yu 2012-04-03, 16:23
+
Stack 2012-04-03, 16:49
+
Nicolas Spiegelberg 2012-04-03, 17:34
+
Stack 2012-04-04, 16:44
+
Ted Yu 2012-04-04, 17:07
+
Stack 2012-04-03, 16:35
+
Joey Echeverria 2012-04-03, 15:01
+
Jean-Daniel Cryans 2012-04-03, 16:43
+
Ted Yu 2012-04-03, 01:10
+
Andrew Purtell 2012-04-03, 00:45