Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Performances Tests


+
Jean-Marc Spaggiari 2013-03-08, 13:58
+
ramkrishna vasudevan 2013-03-08, 14:05
+
Andrew Purtell 2013-03-09, 03:30
+
Jean-Marc Spaggiari 2013-03-13, 00:41
+
Enis Söztutar 2013-03-13, 02:43
+
Jean-Marc Spaggiari 2013-03-17, 02:03
+
lars hofhansl 2013-03-17, 03:23
+
Andrew Purtell 2013-03-17, 10:55
+
Jean-Marc Spaggiari 2013-03-17, 16:47
+
Andrew Purtell 2013-03-17, 17:28
Copy link to this message
-
Re: Performances Tests
Thanks J-M, so that should be easy to reproduce. I'll try.

-- Lars

________________________________
 From: Jean-Marc Spaggiari <[EMAIL PROTECTED]>
To: [EMAIL PROTECTED]
Sent: Sunday, March 17, 2013 9:47 AM
Subject: Re: Performances Tests
 
@Lars: here is an example of what I use
for i in {1..10}; do echo; echo -n $i ; rm -rf /tmp/*;
bin/start-hbase.sh; sleep 60; bin/hbase
org.apache.hadoop.hbase.PerformanceEvaluation sequentialWrite 1;
bin/hbase org.apache.hadoop.hbase.PerformanceEvaluation --rows=100
filterScan 1; bin/stop-hbase.sh; done &>> output.txt

Giving only 1 as the readers allow to not launch the test as a MR. I
have not try the same tests with a bigger number, but I can.

@Andy: LoadTestTool is also on my list, but for YCSB it's a but harder
since I will need more than one dedicated computer. But I will look at
it too. I might be able to remove 1 node from my cluster and dedicate
it to the tests when required...
I will be pretty buzy next week, but before next week-end I will try
to prepare the LoadTestTool scripts to run something similar. With
multiple scenarios, like with and whithout bloom, etc.

If yu have any recommandation/request, feel free! I will come back
soon with some additionnal numbers.

JM

2013/3/17 Andrew Purtell <[EMAIL PROTECTED]>:
> I'm not sure I trust the results of PerformanceEvauation.
>
> LoadTestTool and YCSB have their own issues but seem to produce more
> consistent results. I've been thinking on adding scanning and filtering
> tests to LoadTestTool.
>
> On Sunday, March 17, 2013, lars hofhansl wrote:
>
>> Cool. The 0.94.3 scanning improvements seems almost unbelievable
>> (especially since many of my improvements to reduce the internal friction
>> went into 0.94.4).
>> I would like to track down the random read regression.
>>
>> Can you send the commands you ran? Are you running this as M/R job or
>> standalone client?
>>
>> Thanks for doing this J-M.
>>
>>
>> -- Lars
>>
>>
>>
>> ________________________________
>>  From: Jean-Marc Spaggiari <[EMAIL PROTECTED] <javascript:;>>
>> To: [EMAIL PROTECTED] <javascript:;>
>> Sent: Saturday, March 16, 2013 7:03 PM
>> Subject: Re: Performances Tests
>>
>> Hi Enis,
>>
>> "interesting" in the positive way ;)
>>
>> Results are there:
>>
>> http://www.spaggiari.org/media/blogs/hbase/pictures/performances-1.pdf?mtime=1363484477
>>
>> The improvment on scan are impressive. sequentialRead and randomScan went
>> down.
>>
>> In ran the 0.94.6 tests with RC2. If we have a RC3 I will rerun them.
>>
>> I will add HFilePerformanceEvaluation soon but I'm facinf some issues
>> with it on previous HBase version...
>>
>> JM
>>
>> 2013/3/12 Enis Söztutar <[EMAIL PROTECTED]>:
>> >> I just finished to run all the PerformanceEvaluation tests on a
>> > dedicated computer with all 0.9x.x HBase versions, and I found results
>> > interesting.
>> > Can you please provide your numbers if you can. What is interesting from
>> > your findings?
>> >
>> > Enis
>> >
>> >
>> >
>> > On Tue, Mar 12, 2013 at 5:41 PM, Jean-Marc Spaggiari <
>> > [EMAIL PROTECTED]> wrote:
>> >
>> >> If you run only 1 client with PerformanceEvaluation, it's not running
>> >> it over MapReduce, so you don't have this overhead. But you can still
>> >> run it if you want to have something more distributed. Might be useful
>> >> to have the 2 options. But at the end, LoadTestTool or
>> >> PerformanceEvaluation, any of the 2 is good as long as we are adding
>> >> those tests.
>> >>
>> >> I just finished to run all the PerformanceEvaluation tests on a
>> >> dedicated computer with all 0.9x.x HBase versions, and I found results
>> >> interesting. That gives us a good baseline to see if new HBase
>> >> improvements are really improving performances.
>> >>
>> >> JM
>> >>
>> >> 2013/3/8 Andrew Purtell <[EMAIL PROTECTED]>:
>> >> > Tangentally: I think I prefer LoadTestTool over
>> PerformanceEvaluation, it
>> >> > doesn't depend on nor is influenced by MapReduce job startup.
>> >> >
+
lars hofhansl 2013-03-17, 20:30
+
Ted Yu 2013-03-17, 02:17
+
Jonathan Hsieh 2013-03-19, 18:59
+
Jean-Marc Spaggiari 2013-03-20, 12:02
+
Ted Yu 2013-03-20, 16:08
+
Jean-Marc Spaggiari 2013-03-20, 18:44
+
Enis Söztutar 2013-03-20, 21:29
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB