Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # dev - ycsb and hbase 0.95.1  hadoop2


Copy link to this message
-
Re: ycsb and hbase 0.95.1 hadoop2
Paul Baclace 2013-07-29, 18:50
Thanks for your help, Nicolas. I adjusted the params and I get a
ycsb/db/HBaseClient.java compile error (below) without any missing
package errors, so I'm wondering how it can work for you. I am building
by adding artifacts from 0.95.1 release to my .m2; and you are building
a snapshot and "installing" that to .m2 (or other repo), I assume.

It appears to be missing hbase-common which I added to .m2 and the
compiler debug shows a classpath that only contains
hbase-server-0.95.1-hadoop2.jar (plus ycsb and general jars). Does your
mvn -X compile cmd line for ycsb/db/HBaseClient.java contain more hbase
or hadoop jars? Do you have a settings.xml that makes mvn pull in more
dependencies?
[ERROR]
YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:[34,30]
package org.apache.hadoop.conf does not exist
[ERROR]
/YCSB/hbase/src/main/java/com/yahoo/ycsb/db/HBaseClient.java:[35,31]
cannot find symbol
symbol  : class KeyValue
On 20130729 9:29 , Nicolas Liochon wrote:
> You can use the branch master here: https://github.com/nkeywal/YCSB
> Built it with:
> mvn clean package -DskipTests -Dhbase-96 -Dhbase.version=0.97.0-SNAPSHOT
> -Dhadoop.profile=2.0 -Dhadoop-two.version=2.0.5
> (obviously, change the hbase.version & hadoop-two.version as you need).
>
> I'm awaiting the finalized hadoop2 flags in hbase pom to update this and
> push it to the main ycsb repo.
>
> Cheers,
>
> Nicolas
>
>
> On Mon, Jul 29, 2013 at 6:04 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
>> Hi Paul,
>>
>> I think Nicolas Liochon maintains a personal set of branches for this.
>> Maybe he can point you in the right direction.
>>
>> Thanks,
>> Nick
>>
>> On Mon, Jul 29, 2013 at 12:48 AM, Paul Baclace <[EMAIL PROTECTED]
>>> wrote:
>>> [This might look like a user list question, or a question for another
>>> project, but people on the hbase dev list use ycsb to test hbase, and the
>>> side effect of my progress on this benefits hbase more than any other
>>> project... please indulge me.]
>>>
>>> I have been unable to build ycsb for hbase 0.95.1  hadoop2, even though I
>>> added the artifacts to my local repo like so:
>>>
>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>> hadoop2/lib/hadoop-mapreduce-**client-core-2.0.2-alpha.jar
>>> -DgroupId=org.apache.hadoop -DartifactId=hadoop-mapreduce-**client-core
>>> -Dversion=2.0.2-alpha -Dpackaging=jar
>>>
>>> mvn install:install-file -Dfile=../hbase-0.95.1-**
>>> hadoop2/lib/hbase-client-0.95.**1-hadoop2.jar -DgroupId=org.apache.hbase
>>> -DartifactId=hbase-client -Dversion=0.95.1-hadoop2 -Dpackaging=jar
>>>
>>> I looked at the pom.xml in the jar files to determine the parameters,
>>> modified ycsb pom.xml and hbase/pom.xml; ycsb  "mvn clean package" has
>>> errors, not finding basic stuff:
>>>
>>> [ERROR] YCSB/hbase/src/main/**
>>> java/com/yahoo/ycsb/db/**HBaseClient.java:[34,29] package
>>> org.apache.hadoop.conf does not exist
>>> [ERROR] YCSB/hbase/src/main/**
>>> java/com/yahoo/ycsb/db/**HBaseClient.java:[35,30] cannot find symbol
>>> symbol  : class KeyValue
>>> location: package org.apache.hadoop.hbase
>>> [ERROR] YCSB/hbase/src/main/**
>>> java/com/yahoo/ycsb/db/**HBaseClient.java:[46,35] cannot find symbol
>>> symbol  : class Bytes
>>> location: package org.apache.hadoop.hbase.util
>>> [ERROR] YCSB/hbase/src/main/**
>>> java/com/yahoo/ycsb/db/**HBaseClient.java:[47,30] cannot find symbol
>>> symbol  : class HBaseConfiguration
>>> location: package org.apache.hadoop.hbase
>>> [... more not shown for brevity ]
>>>
>>> I suspect ycsb pom.xml [https://github.com/**brianfrankcooper/YCSB<
>> https://github.com/brianfrankcooper/YCSB>]
>>> is not ready for packaging changes made in hbase 0.95 OR could it be that
>>> adding jars to a local maven repo, because they are not yet on a
>>> centralized server, simply does not suffice because not all dependencies
>>> are added?   HBaseConfiguration is in hbase-common-0.95.1-hadoop2.**jar ,
>>> but how to add all the turtles all the way down?
>>>
>>