Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Re: small perf degradation in 0.94 trunk vs. older versions


Copy link to this message
-
Re: small perf degradation in 0.94 trunk vs. older versions
Here are some tests on a standalone hbase. I just create a table with 3000
regions from the shell: create 't1', 'f1', {NUMREGIONS => 3000, SPLITALGO
=> 'HexStringSplit'}

It seems that's it's between august 20th and september 1st, but there could
be some small other stuff along the line as well...

commit eb36a3e410998e72d76edcc5c181dfa54bba1c39
Date:   Tue Oct 2 20:39:14 2012 +0000
0 row(s) in 184.4040 seconds
0 row(s) in 191.4830 seconds
0 row(s) in 189.6370 seconds
0 row(s) in 198.9080 seconds
0 row(s) in 202.8180 seconds

commit 8c93fcfca2eb12162b99b8e1e327bab872bba6b7
Date:   Wed May 30 17:06:52 2012 +0000
create 't1', 'f1', {NUMREGIONS => 3000, SPLITALGO => 'HexStringSplit'}
0 row(s) in 165.2160 seconds
0 row(s) in 142.7890 seconds
0 row(s) in 137.7260 seconds

commit 01f0b3c77ea30b1c22a0e96929e97bb9f5faab2bgit
Date:   Thu Jul 19 22:19:49 2012 +0000
0 row(s) in 173.9160 seconds
0 row(s) in 155.0360 seconds
0 row(s) in 141.5210 seconds
0 row(s) in 140.6480 seconds

commit bc51a7267d2647630224f646b62b80458591414a
Date:   Sat Sep 1 04:35:29 2012 +0000
0 row(s) in 161.1990 seconds
0 row(s) in 171.0010 seconds
0 row(s) in 167.2430 seconds
0 row(s) in 189.9960 seconds
0 row(s) in 205.0460 seconds
0 row(s) in 194.4270 seconds
0 row(s) in 191.9020 seconds

commit 5ea73b7992819ec6e24ba7c2c5beee306f7d12da
Date:   Thu Aug 9 21:52:21 2012 +0000
0 row(s) in 164.1430 seconds
0 row(s) in 154.3410 seconds
0 row(s) in 154.3370 seconds
0 row(s) in 142.6060 seconds
0 row(s) in 138.8110 seconds
0 row(s) in 144.4190 seconds

commit 7179970b4a00ce630004d72e8e45e30fa9f4881b
Date:   Mon Aug 20 23:38:50 2012 +0000
0 row(s) in 146.4550 seconds
0 row(s) in 150.3020 seconds
0 row(s) in 143.6050 seconds
0 row(s) in 148.4000 seconds
0 row(s) in 164.8320 seconds

commit b4e5c3ae45935a37e1ab8651998c6a8131180eec
Date:   Mon Aug 13 19:32:19 2012 +0000
0 row(s) in 151.0200 seconds
0 row(s) in 186.2860 seconds
0 row(s) in 141.6130 seconds
0 row(s) in 141.1880 seconds
0 row(s) in 140.9320 seconds
0 row(s) in 162.9540 seconds
0 row(s) in 139.5530 seconds
On Thu, Nov 29, 2012 at 10:07 AM, Nicolas Liochon <[EMAIL PROTECTED]> wrote:

> I'm going to try with some random commits. Hopefully I will reproduce it a
> on standalone instance as well...
> Stay tuned :-)
>
>
>
> On Wed, Nov 28, 2012 at 11:02 PM, lars hofhansl <[EMAIL PROTECTED]>wrote:
>
>> Did a quick scan through the changes committed since 0.94.1:
>> HBASE-6608
>> HBASE-6364
>> HBASE-6587
>> HBASE-6537
>> HBASE-6713
>> HBASE-6438
>> HBASE-6299
>> HBASE-7018
>> HBASE-7038
>> HBASE-7060
>>
>> Any chance you can do this again with 0.94.2 (or even do a binary search
>> through the commits to pinpoint the change)?
>>
>> -- Lars
>>
>>   ------------------------------
>> *From:* Nicolas Liochon <[EMAIL PROTECTED]>
>> *To:* [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
>> *Sent:* Tuesday, November 27, 2012 10:42 AM
>> *Subject:* Re: small perf degradation in 0.94 trunk vs. older versions
>>
>> It was Version 0.94.3, r38dbd22c99debd9010e9e5f4fbabeeaf3c4e1ddd
>>
>>
>> On Tue, Nov 27, 2012 at 7:41 PM, lars hofhansl <[EMAIL PROTECTED]>wrote:
>>
>> Interesting. Thanks N.
>>
>> I'll look through the 0.94 commits since 0.94.1 to see what's causing
>> this.
>> With 0.94 trunk you mean the 0.94.3RC?
>>
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>>  From: Nicolas Liochon <[EMAIL PROTECTED]>
>> To: [EMAIL PROTECTED]
>> Sent: Tuesday, November 27, 2012 5:07 AM
>> Subject: small perf degradation in 0.94 trunk vs. older versions
>>
>> Hi,
>>
>> On a create table / reassignment, I feel we lost some performances
>> recently
>> on 0.94. It's not huge (5% to 10%), but I would prefer them to be the
>> other
>> way around.
>> Here are some tests I've done on test cluster, all the time with Hadoop
>> 1.1.0;
>>
>> scenario: 2 RS. Create a table with 3000 regions.
>> Start a third RS. Kill -15 the second one: there are 1500 regions to
>> reassign.
>>
>> I've made multiple measures, there are all there: