Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> 30% random performance in 0.95+


Copy link to this message
-
Re: 30% random performance in 0.95+
In my measurements 0.94 has been getting faster with each release in both read and write performance.
I wonder how representative PE is after all; it only tests via the local FS layer (not HDFS), among other issues.

-- Lars

----- Original Message -----
From: Jean-Marc Spaggiari <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]; lars hofhansl <[EMAIL PROTECTED]>
Cc:
Sent: Friday, June 28, 2013 8:03 PM
Subject: Re: 30% random performance in 0.95+

I think we should do that on 0.94 as well. I don't see any good reason
to not do it.

JM

2013/6/28 lars hofhansl <[EMAIL PROTECTED]>:
> Yep.
> Now the question is: Make these changes to 0.94 as well? Or just document these better.
>
> -- Lars
>
>
>
> ----- Original Message -----
> From: Andrew Purtell <[EMAIL PROTECTED]>
> To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>; lars hofhansl <[EMAIL PROTECTED]>
> Cc:
> Sent: Friday, June 28, 2013 2:08 PM
> Subject: Re: 30% random performance in 0.95+
>
> I've been thinking about how to periodically search through some of our
> parameter space to see what changes to defaults are better all the way
> around. Probably will so something based on Bigtop.
>
>
> On Friday, June 28, 2013, lars hofhansl wrote:
>
>> And indeed just this makes a tremendous difference. Unpatched 0.94 with
>> 40% block cache configured is actually faster than 0.95 with the same block
>> cache size.
>>
>> -- Lars
>>
>>
>>
>> ----- Original Message -----
>> From: lars hofhansl <[EMAIL PROTECTED] <javascript:;>>
>> To: "[EMAIL PROTECTED] <javascript:;>" <[EMAIL PROTECTED]<javascript:;>
>> >
>> Cc:
>> Sent: Friday, June 28, 2013 1:34 PM
>> Subject: Re: 30% random performance in 0.95+
>>
>> Thanks JM,
>>
>> HBASE-8450 (r1485562) is interesting. It increases (among other things)
>> the block cache percentage from 24 to 40%, which would lead to a higher
>> probability of a future random read to hit an already cached block.
>>
>>
>> -- Lars
>>
>>
>>
>> ----- Original Message -----
>> From: Jean-Marc Spaggiari <[EMAIL PROTECTED] <javascript:;>>
>> To: [EMAIL PROTECTED] <javascript:;>; lars hofhansl <[EMAIL PROTECTED]<javascript:;>
>> >
>> Cc:
>> Sent: Friday, June 28, 2013 1:18 PM
>> Subject: Re: 30% random performance in 0.95+
>>
>> I have the script done to run over a list of "svn releases", so if
>> required, just give me a bunch of them or a range and I can restart.
>> Just keep me posted.
>>
>> JM
>>
>> 2013/6/28 lars hofhansl <[EMAIL PROTECTED] <javascript:;>>:
>> > I did a few more test (on my laptop, which is not quite representative),
>> and found only a 2-3% improvement from HBASE-8001+HBASE-8012 in the end.
>> > I'll look through the issues that you identified.
>> >
>> > -- Lars
>> >
>> >
>> >
>> > ----- Original Message -----
>> > From: Jean-Marc Spaggiari <[EMAIL PROTECTED] <javascript:;>>
>> > To: [EMAIL PROTECTED] <javascript:;>
>> > Cc:
>> > Sent: Friday, June 28, 2013 12:51 PM
>> > Subject: Re: 30% random performance in 0.95+
>> >
>> > Sorry folks,
>> >
>> > I'm a bit late to run the tests... 0.94.8 and 0.94.9 are currently
>> > running, but here is what I have been able to capture so far for 0.95
>> > over the last year:
>> > r1357480 1513196
>> > r1367009 1440244.4
>> > r1375812 1287143.5
>> > r1381671 1287200.2
>> > r1388620 1295262.6
>> > r1394335 1022140.2
>> > r1403898 884171.9
>> > r1410631 804229.9
>> > r1419787 846816.9
>> > r1426557 853535.3
>> > r1433514 873265.1
>> > r1438972 840666.9
>> > r1446106 877432.2
>> > r1452661 883974.8
>> > r1458421 882233.3
>> > r1464267 847000.8
>> > r1478964 877433.5
>> > r1485868 744905.5
>> > r1494869 765105.9
>> >
>> > So seems that there was some improvements between r1367009 and
>> > r1403898 but they are old. Also another major improvement between
>> > r1478964 and r1485868...
>> >
>> > Let me know if you want me to dig further and I will be very happy to do
>> so.
>> >
>> > JM
>> >
>> > 2013/6/28 Stack <[EMAIL PROTECTED] <javascript:;>>:
>> >> On Fri, Jun 28, 2013 at 10:53 AM, lars hofhansl <[EMAIL PROTECTED]<javascript:;>>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB