Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> ycsb and hbase 0.95.1  hadoop2


+
Paul Baclace 2013-07-29, 07:48
+
Nick Dimiduk 2013-07-29, 16:04
+
Nicolas Liochon 2013-07-29, 16:29
+
Paul Baclace 2013-07-29, 18:50
+
Nicolas Liochon 2013-07-29, 19:18
+
Paul Baclace 2013-07-29, 20:11
+
Ted Yu 2013-07-29, 20:14
+
Nicolas Liochon 2013-07-29, 20:34
Copy link to this message
-
Re: ycsb and hbase 0.95.1 hadoop2
I eventually determined that bin/ycsb (a python script) was causing most
of the linkage problems, so I recommend using your own bash script to
start ycsb for HBase. The problem is that the ordering of paths in the
classpath is completely uncontrolled: bin/ycsb does not put the
hbase-binding/conf first, so it can pick up a default hbase-site.xml
found in a jar, and it will recursively include any jars in any
subdirectories under hbase-binding/ (a surprise when I mv'ed some files
to a subdir like dont_use_these and lsof showed them to be in use).

I ended up using hbase 0.95.1 hadoop1 on Apache Hadoop 1.0.4 for my
benchmarking.
On 20130729 13:34 , Nicolas Liochon wrote:
> 0.95.2 is not yet finished. As Ted suggested, you can build HBase locally
> from the sources (the 0.95 branch is usually good as the patches are tested
> before being committed).
>
> When 0.95.2 will be out I will redo the integration with ycsb. Hopefully it
> will be simple to make it work with all hadoop versions on the standards
> maven repo...
>
>
> On Mon, Jul 29, 2013 at 10:14 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
>> You can perform the following command in 0.95 workspace:
>>
>> mvn clean install -DskipTests
>>
>> Cheers
>>
>> On Mon, Jul 29, 2013 at 1:11 PM, Paul Baclace <[EMAIL PROTECTED]
>>> wrote:
>>> On 20130729 12:18 , Nicolas Liochon wrote:
>>>
>>>> No, I've got nothing in my setting.xml
>>>> I build HBase locally (mvn install -Dhadoop.profile=2.0 -DskipTests), so
>>>> yes, it goes into my .m2
>>>>
>>>>
>>>> I've just rechecked with the last 0.97 as of now, it does compile. It
>>>> worked as well 2 weeks ago with the 0.95 branch. I'm not sure about the
>>>> 95.1. I remember some nasty maven issues.
>>>>
>>> ((I rather be attacked by angry ants than nasty mavens.))
>>>
>>>
>>>> If you're using the maven repo for the 0.95.1, IIRC there is no version
>>>> available that works well with hadoop 2 with a maven-built client (it's
>> ok
>>>> if you build hbase locally with the right hadoop version).
>>>>
>>> Is there an experimental or nightly 0.95.2 maven repo I can connect to? I
>>> can probably use 0.95.2 if all the tests passed; it looks like
>>> https://builds.apache.org/job/**HBase-0.95/373<
>> https://builds.apache.org/job/HBase-0.95/373>passed.
>>>
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>> On Mon, Jul 29, 2013 at 8:50 PM, Paul Baclace <[EMAIL PROTECTED]
>>>>> wrote:
>>>>   Thanks for your help, Nicolas. I adjusted the params and I get a
>>>>> ycsb/db/HBaseClient.java compile error (below) without any missing
>>>>> package
>>>>> errors, so I'm wondering how it can work for you. I am building by
>> adding
>>>>> artifacts from 0.95.1 release to my .m2; and you are building a
>> snapshot
>>>>> and "installing" that to .m2 (or other repo), I assume.
>>>>>
>>>>> It appears to be missing hbase-common which I added to .m2 and the
>>>>> compiler debug shows a classpath that only contains
>>>>> hbase-server-0.95.1-hadoop2.****jar (plus ycsb and general jars). Does
>>>>> your
>>>>> mvn -X compile cmd line for ycsb/db/HBaseClient.java contain more hbase
>>>>> or
>>>>> hadoop jars? Do you have a settings.xml that makes mvn pull in more
>>>>> dependencies?
>>>>>
>>>>>
>>>>> [ERROR] YCSB/hbase/src/main/java/com/****yahoo/ycsb/db/HBaseClient.****
>>>>> java:[34,30]
>>>>> package org.apache.hadoop.conf does not exist
>>>>> [ERROR]
>> /YCSB/hbase/src/main/java/com/****yahoo/ycsb/db/HBaseClient.****
>>>>> java:[35,31]
>>>>> cannot find symbol
>>>>> symbol  : class KeyValue
>>>>>
>>>>>
>>>>>
>>>>> On 20130729 9:29 , Nicolas Liochon wrote:
>>>>>
>>>>>   You can use the branch master here:
>> https://github.com/nkeywal/****YCSB<https://github.com/nkeywal/**YCSB>
>>>>>> <https://github.com/**nkeywal/YCSB <https://github.com/nkeywal/YCSB>>
>>>>>> Built it with:
>>>>>> mvn clean package -DskipTests -Dhbase-96 -Dhbase.version=0.97.0-**
>>>>>> SNAPSHOT
>>>>>> -Dhadoop.profile=2.0 -Dhadoop-two.version=2.0.5
>>>>>> (obviously, change the hbase.version & hadoop-two.version as you
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB