Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> Guidance for running MR jobs against trunk/0.95


+
Nick Dimiduk 2013-02-25, 22:37
+
Nick Dimiduk 2013-02-25, 22:58
+
Elliott Clark 2013-02-25, 23:10
Copy link to this message
-
Re: Guidance for running MR jobs against trunk/0.95
At least. What about the hadoop*-compat modules? I can reference explicit
class names for the other modules, how would you recommend pulling in
references from the appropriate compat module?

[INFO] --- maven-dependency-plugin:2.1:tree (default-cli) @ hbase-server ---
[INFO] org.apache.hbase:hbase-server:jar:0.95-SNAPSHOT
[INFO] +- org.apache.hbase:hbase-common:jar:0.95-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-protocol:jar:0.95-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-client:jar:0.95-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-prefix-tree:jar:0.95-SNAPSHOT:runtime
[INFO] +- org.apache.hbase:hbase-hadoop-compat:jar:0.95-SNAPSHOT:compile
[INFO] +- org.apache.hbase:hbase-hadoop1-compat:jar:0.95-SNAPSHOT:compile

On Mon, Feb 25, 2013 at 3:10 PM, Elliott Clark <[EMAIL PROTECTED]> wrote:

> You'll currently need:
>
> hbase-server
> hbase-common
> hbase-protocol
> hbase-prefix-tree  (after HBASE-7188)
> hbase-client (after HBASE-7188)
>
> on your classpath to do any mapreduce jobs.
>
>
> On Mon, Feb 25, 2013 at 2:37 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
> >
> > I'm seeing ClassNotFoundExceptions when running ImportTsv against trunk.
> > Presumably the old instructions for classpath assembly need an update. I
> > guess I could create a hadoop jar that ships the classpath entries in
> lib,
> > but IMHO that's not a viable solution for users who want to run one of
> the
> > jobs we distribute. Am I missing something?
> >
> > Thanks,
> > Nick
> >
> > For you reference:
> >
> > $ HADOOP_CLASSPATH=$(./bin/hbase classpath) hadoop jar
> > ./hbase-server/target/hbase-server-0.95-SNAPSHOT.jar importtsv
> > -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2
> > -Dimporttsv.bulk.output=hdfs:///tmp/hfiles simple hdfs:///tmp/simple.tsv
> >
> > ...
> >
> > Error: java.lang.ClassNotFoundException:
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Mutate$MutateType
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> >  at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > at
> >
> >
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:87)
> >  at
> >
> >
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:70)
> > at
> >
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1069)
> >  at
> >
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
> > at
> >
> >
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> >  at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:151)
> > at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:37)
> >  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> >  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> > at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >  at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:396)
> >  at
> >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> > at org.apache.hadoop.mapred.Child.main(Child.java:249)
> >
>
+
Elliott Clark 2013-02-25, 23:27
+
Nick Dimiduk 2013-02-25, 23:54
+
Matt Corgan 2013-02-26, 00:01
+
Nick Dimiduk 2013-02-26, 00:41
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB