Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - Guidance for running MR jobs against trunk/0.95


+
Nick Dimiduk 2013-02-25, 22:37
+
Nick Dimiduk 2013-02-25, 22:58
+
Elliott Clark 2013-02-25, 23:10
+
Nick Dimiduk 2013-02-25, 23:19
+
Elliott Clark 2013-02-25, 23:27
Copy link to this message
-
Re: Guidance for running MR jobs against trunk/0.95
Nick Dimiduk 2013-02-25, 23:54
Under what context is hbase-prefix-tree necessary? Currently, this
dependency is scoped as 'runtime'. Adding it explicitly will require this
be changed.

Thanks,
Nick

On Mon, Feb 25, 2013 at 3:10 PM, Elliott Clark <[EMAIL PROTECTED]> wrote:

> You'll currently need:
>
> hbase-server
> hbase-common
> hbase-protocol
> hbase-prefix-tree  (after HBASE-7188)
> hbase-client (after HBASE-7188)
>
> on your classpath to do any mapreduce jobs.
>
>
> On Mon, Feb 25, 2013 at 2:37 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
> >
> > I'm seeing ClassNotFoundExceptions when running ImportTsv against trunk.
> > Presumably the old instructions for classpath assembly need an update. I
> > guess I could create a hadoop jar that ships the classpath entries in
> lib,
> > but IMHO that's not a viable solution for users who want to run one of
> the
> > jobs we distribute. Am I missing something?
> >
> > Thanks,
> > Nick
> >
> > For you reference:
> >
> > $ HADOOP_CLASSPATH=$(./bin/hbase classpath) hadoop jar
> > ./hbase-server/target/hbase-server-0.95-SNAPSHOT.jar importtsv
> > -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2
> > -Dimporttsv.bulk.output=hdfs:///tmp/hfiles simple hdfs:///tmp/simple.tsv
> >
> > ...
> >
> > Error: java.lang.ClassNotFoundException:
> > org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Mutate$MutateType
> > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> >  at java.security.AccessController.doPrivileged(Native Method)
> > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> >  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > at
> >
> >
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:87)
> >  at
> >
> >
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:70)
> > at
> >
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1069)
> >  at
> >
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
> > at
> >
> >
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> >  at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:151)
> > at
> >
> >
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:37)
> >  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> >  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> > at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> >  at java.security.AccessController.doPrivileged(Native Method)
> > at javax.security.auth.Subject.doAs(Subject.java:396)
> >  at
> >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> > at org.apache.hadoop.mapred.Child.main(Child.java:249)
> >
>
+
Matt Corgan 2013-02-26, 00:01
+
Nick Dimiduk 2013-02-26, 00:41