Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> [VOTE] The 1st hbase 0.94.19 release candidate is available for download


Copy link to this message
-
Re: [VOTE] The 1st hbase 0.94.19 release candidate is available for download
Wow. Thanks JM!

________________________________
From: Jean-Marc Spaggiari <[EMAIL PROTECTED]>
To: dev <[EMAIL PROTECTED]>
Sent: Thursday, April 24, 2014 10:09 AM
Subject: Re: [VOTE] The 1st hbase 0.94.19 release candidate is available for download
So. I'm done with my heavy duty release test for 0.94.19.

tl.tr: +1 ;)

Here are the details.
Downloaded the jar, checked the signature, the CHANGES.txt file, the
documentation (random pickup) -> Passed.
Run the test suite -> Passed.
Tests run: 1550, Failures: 0, Errors: 0, Skipped: 16
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 1:16:23.687s
[INFO] Finished at: Wed Apr 23 12:02:26 EDT 2014
[INFO] Final Memory: 29M/983M
[INFO]

Run PE over few days and compared with 0.94.18.
Got one exception on GaussianRandomReadBenchmark for
HFilePerformanceEvaluation. This occurs multiple times but was always the
same exception:
org.apache.hadoop.hbase.io.hfile.AbstractHFileReader$NotSeekedException:
Not seeked to a key/value
        at
org.apache.hadoop.hbase.io.hfile.AbstractHFileReader$Scanner.assertSeeked(AbstractHFileReader.java:320)
        at
org.apache.hadoop.hbase.io.hfile.HFileReaderV2$ScannerV2.getKey(HFileReaderV2.java:650)
        at
org.apache.hadoop.hbase.HFilePerformanceEvaluation$GaussianRandomReadBenchmark.doRow(HFilePerformanceEvaluation.java:350)
        at
org.apache.hadoop.hbase.HFilePerformanceEvaluation$RowOrientedBenchmark.run(HFilePerformanceEvaluation.java:169)
        at
org.apache.hadoop.hbase.HFilePerformanceEvaluation.runBenchmark(HFilePerformanceEvaluation.java:121)
        at
org.apache.hadoop.hbase.HFilePerformanceEvaluation$3.run(HFilePerformanceEvaluation.java:97)
        at java.lang.Thread.run(Thread.java:744)

Performance wise this release is very similar to the previous one (>0% mean
0.94.19 is faster):

0.94.180.94.19
FilteredScanTest0,23 0,230,01%RandomReadTest 8258250,01% RandomSeekScanTest
173178 2,89%RandomScanWithRange10Test282 2861,58%RandomScanWithRange100Test149
147-1,10% RandomScanWithRange1000Test37,5739,40
4,85%SequentialReadTest1 2061 225
1,51%SequentialWriteTest 13 68713 8261,02% RandomWriteTest14 09213 574-3,68%
GaussianRandomReadBenchmark9 400 9 395-0,05%SequentialReadBenchmark3 035 361
3 009 210-0,86% SequentialWriteBenchmark909 881909 579 -0,03%
UniformRandomReadBenchmark10 312 10 3540,41%UniformRandomSmallScan 233 141
233 9310,34% LoadTestToolreal 19m24.218s
user 36m17.208s
sys 11m45.128sreal 19m11.070s
user 37m5.328s
sys 11m10.724s
IntegrationTestLoadAnVerify real 4m7.922s
user 1m31.100s
sys 0m7.324sreal 4m7.909s
user 1m30.668s
sys 0m7.136s
HLogPerformanceEvaluation10571,8310597,149
0,24%IntegrationTestBigLinkedListreal
6m20.893s
user 2m55.068s
sys 0m10.728s real 6m26.026s
user 3m0.156s
sys 0m10.436s

Ran LoadTestTool, IntegrationTestLoadAndVerify, HLogPerformanceEvaluation,
IntegrationTestBigLinkedList. I consider them as passed since I have the
same results as for 0.94.18 but I still have the
IntegrationTestLoadAndVerify reporting wrong results.

14/04/23 22:45:03 INFO mapred.JobClient:
org.apache.hadoop.hbase.test.IntegrationTestLoadAndVerify$Counters
14/04/23 22:45:03 INFO mapred.JobClient:     ROWS_WRITTEN=0
14/04/23 22:45:03 INFO mapred.JobClient:     REFERENCES_CHECKED=9855522
(Should be 10000000)

Tried to manually create table, put, scan, flush, get, compact, drop, etc.
All passed.

Deployed on 8 node cluster. RSs rolling restart went well.
Stopped the cluster, merged a 32 region table to a single region, restart,
major_compact, got it splitted correctly.

Tried the default load balancer but did not worked well the first time.
Initial state:
node7.google.com <http://node7.distparser.com>,60020,1398349950181     32

After first run:
Regions by Region Server
Region Server    Region Count
node1. <http://node1.distparser.com>google
<http://node7.distparser.com>.com<http://node1.distparser.com>,60020,1398349948960
    2
node2. <http://node2.distparser.com>google
<http://node7.distparser.com>.com<http://node2.distparser.com>,60020,1398349952018
    3
node3. <http://node3.distparser.com>google
<http://node7.distparser.com>.com<http://node3.distparser.com>,60020,1398349947506
    2
node4. <http://node4.distparser.com>google
<http://node7.distparser.com>.com<http://node4.distparser.com>,60020,1398349949141
    3
node5. <http://node5.distparser.com>google
<http://node7.distparser.com>.com<http://node5.distparser.com>,60020,1398349946262
    2
node6. <http://node6.distparser.com>google
<http://node7.distparser.com>.com<http://node6.distparser.com>,60020,1398349951095
    2
node7. <http://node7.distparser.com>google
<http://node7.distparser.com>.com<http://node7.distparser.com>,60020,1398349950181
    16
node8. <http://node8.distparser.com>google
<http://node7.distparser.com>.com<http://node8.distparser.com>,60020,1398349948480
    2

After 2nd run:
Regions by Region Server
Region Server    Region Count
node1. <http://node1.distparser.com>google
<http://node7.distparser.com>.com<http://node1.distparser.com>,60020,1398349948960
    4
node2. <http://node2.distparser.com>google
<http://node7.distparser.com>.com<http://node2.distparser.com>,60020,1398349952018
    4
node3. <http://node3.distparser.com>google
<http://node7.distparser.com>.com<http://node3.distparser.com>,60020,1398349947506
    4
node4. <http://node4.distparser.com>google
<http://node7.distparser.com>.com<http://node4.distparser.com>,60020,1398349949141
    4
node5. <http://node5.distparser.com>google
<http://node7.distparser.com>.com<http://node5.distparser.com>,60020,1398349946262
    4
node6. <http://node6.distparser.com>google
<http://node7.distparser.com>.com<http://node6.distparser.com>,60020,1398349951095
    4
node7. <http://node7.distparser.com>google
<http://node7.distparser.com>.com<http://node7.distparser.com>,60020,1398349950181
    4
node8. <http://node8.distparser.com>google
<http://node7.distparser.com>.com<http://node8.distparser.com>,60020,1398349948480
    4

But that's not a show stopper.
Restor