Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - Performances Tests


Copy link to this message
-
Re: Performances Tests
Jean-Marc Spaggiari 2013-03-20, 18:44
Hi Ted,

I will try to build the trunk version and add it in the list....

So I have to:
- Re-download all HBase version to make sure PE is using the right one;
- Re-run the scanRange100 tests to validate the values already found;
- Add the trunk;
- Add LoadTestTool

Than will keep my free time buzy ;)

I keep you all posted as soon as it's done.

JM

2013/3/20 Ted Yu <[EMAIL PROTECTED]>:
> I am curious to know how trunk stands in the performance comparison.
> There have been many optimizations going into trunk. Getting hold of
> overall improvement would be nice.
>
> Cheers
>
> On Wed, Mar 20, 2013 at 5:02 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Hi Lars,
>>
>> Can you share the code you are using so I can compate with PE? Also, I
>> will re-run all for my scanRange100 tests today and update the
>> spreadsheet again to make sure it's correct. Also also re-download all
>> the HBase versions to make sure they are all clean. I'm not doing any
>> configuration with them. Simply reducing the logs and tmp pointing to
>> memory file system.
>>
>> I will keep you posted when it's done.
>>
>> Hi Jonathan,
>>
>> It's usually rows per seconds, but with a factor 10. Sometime I had to
>> divide by 100000, sometime to multiply to get numbers bigger... I will
>> take a look at th formulas and add the legend for each of the charts.
>>
>> JM
>>
>> 2013/3/19 Jonathan Hsieh <[EMAIL PROTECTED]>:
>> > What is the y axis's unit?  seconds or operations per second etc?  (nit:
>> > would be nice to have on the axis.. )
>> >
>> > Based on the context, I believe it is ops/s.
>> >
>> > Jon.
>> >
>> > On Sat, Mar 16, 2013 at 7:03 PM, Jean-Marc Spaggiari <
>> > [EMAIL PROTECTED]> wrote:
>> >
>> >> Hi Enis,
>> >>
>> >> "interesting" in the positive way ;)
>> >>
>> >> Results are there:
>> >>
>> >>
>> http://www.spaggiari.org/media/blogs/hbase/pictures/performances-1.pdf?mtime=1363484477
>> >>
>> >> The improvment on scan are impressive. sequentialRead and randomScan
>> went
>> >> down.
>> >>
>> >> In ran the 0.94.6 tests with RC2. If we have a RC3 I will rerun them.
>> >>
>> >> I will add HFilePerformanceEvaluation soon but I'm facinf some issues
>> >> with it on previous HBase version...
>> >>
>> >> JM
>> >>
>> >> 2013/3/12 Enis Söztutar <[EMAIL PROTECTED]>:
>> >> >> I just finished to run all the PerformanceEvaluation tests on a
>> >> > dedicated computer with all 0.9x.x HBase versions, and I found results
>> >> > interesting.
>> >> > Can you please provide your numbers if you can. What is interesting
>> from
>> >> > your findings?
>> >> >
>> >> > Enis
>> >> >
>> >> >
>> >> >
>> >> > On Tue, Mar 12, 2013 at 5:41 PM, Jean-Marc Spaggiari <
>> >> > [EMAIL PROTECTED]> wrote:
>> >> >
>> >> >> If you run only 1 client with PerformanceEvaluation, it's not running
>> >> >> it over MapReduce, so you don't have this overhead. But you can still
>> >> >> run it if you want to have something more distributed. Might be
>> useful
>> >> >> to have the 2 options. But at the end, LoadTestTool or
>> >> >> PerformanceEvaluation, any of the 2 is good as long as we are adding
>> >> >> those tests.
>> >> >>
>> >> >> I just finished to run all the PerformanceEvaluation tests on a
>> >> >> dedicated computer with all 0.9x.x HBase versions, and I found
>> results
>> >> >> interesting. That gives us a good baseline to see if new HBase
>> >> >> improvements are really improving performances.
>> >> >>
>> >> >> JM
>> >> >>
>> >> >> 2013/3/8 Andrew Purtell <[EMAIL PROTECTED]>:
>> >> >> > Tangentally: I think I prefer LoadTestTool over
>> >> PerformanceEvaluation, it
>> >> >> > doesn't depend on nor is influenced by MapReduce job startup.
>> >> >> >
>> >> >> >
>> >> >> > On Fri, Mar 8, 2013 at 10:05 PM, ramkrishna vasudevan <
>> >> >> > [EMAIL PROTECTED]> wrote:
>> >> >> >
>> >> >> >> @JM
>> >> >> >> I agree with you.  Mainly the perf improvement changes needs some
>> >> >> >> testcases.
>> >> >> >> But sometimes the scenario on which the perf improvments happens