Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # dev - Guidance for running MR jobs against trunk/0.95


+
Nick Dimiduk 2013-02-25, 22:37
Copy link to this message
-
Re: Guidance for running MR jobs against trunk/0.95
Nick Dimiduk 2013-02-25, 22:58
Actually, I think this is a bug in TableMapReduceUtil. Opened HBASE-7934.

Thanks,
Nick

On Mon, Feb 25, 2013 at 2:37 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I'm seeing ClassNotFoundExceptions when running ImportTsv against trunk.
> Presumably the old instructions for classpath assembly need an update. I
> guess I could create a hadoop jar that ships the classpath entries in lib,
> but IMHO that's not a viable solution for users who want to run one of the
> jobs we distribute. Am I missing something?
>
> Thanks,
> Nick
>
> For you reference:
>
> $ HADOOP_CLASSPATH=$(./bin/hbase classpath) hadoop jar
> ./hbase-server/target/hbase-server-0.95-SNAPSHOT.jar importtsv
> -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2
> -Dimporttsv.bulk.output=hdfs:///tmp/hfiles simple hdfs:///tmp/simple.tsv
>
> ...
>
> Error: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Mutate$MutateType
> at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>  at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> at
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:87)
>  at
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:70)
> at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1069)
>  at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
> at
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
>  at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:151)
> at
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:37)
>  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
>  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
>  at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:396)
>  at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> at org.apache.hadoop.mapred.Child.main(Child.java:249)
>
+
Elliott Clark 2013-02-25, 23:10
+
Nick Dimiduk 2013-02-25, 23:19
+
Elliott Clark 2013-02-25, 23:27
+
Nick Dimiduk 2013-02-25, 23:54
+
Matt Corgan 2013-02-26, 00:01
+
Nick Dimiduk 2013-02-26, 00:41