Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # dev >> Guidance for running MR jobs against trunk/0.95


Copy link to this message
-
Re: Guidance for running MR jobs against trunk/0.95
Right now only used at runtime on the server.  I guess we need a way to
prevent instantiation on the client.

If we allowed it as one of the client RPC encodings, then client would need
the module as well.  However, I don't think prefix-tree is a good candidate
for client RPC encoding anyway since RPC is a streaming activity, not
random access.  The existing delta encoders will offer higher performance
with similar bandwidth savings.
On Mon, Feb 25, 2013 at 3:54 PM, Nick Dimiduk <[EMAIL PROTECTED]> wrote:

> Under what context is hbase-prefix-tree necessary? Currently, this
> dependency is scoped as 'runtime'. Adding it explicitly will require this
> be changed.
>
> Thanks,
> Nick
>
> On Mon, Feb 25, 2013 at 3:10 PM, Elliott Clark <[EMAIL PROTECTED]> wrote:
>
> > You'll currently need:
> >
> > hbase-server
> > hbase-common
> > hbase-protocol
> > hbase-prefix-tree  (after HBASE-7188)
> > hbase-client (after HBASE-7188)
> >
> > on your classpath to do any mapreduce jobs.
> >
> >
> > On Mon, Feb 25, 2013 at 2:37 PM, Nick Dimiduk <[EMAIL PROTECTED]>
> wrote:
> >
> > > Hi all,
> > >
> > > I'm seeing ClassNotFoundExceptions when running ImportTsv against
> trunk.
> > > Presumably the old instructions for classpath assembly need an update.
> I
> > > guess I could create a hadoop jar that ships the classpath entries in
> > lib,
> > > but IMHO that's not a viable solution for users who want to run one of
> > the
> > > jobs we distribute. Am I missing something?
> > >
> > > Thanks,
> > > Nick
> > >
> > > For you reference:
> > >
> > > $ HADOOP_CLASSPATH=$(./bin/hbase classpath) hadoop jar
> > > ./hbase-server/target/hbase-server-0.95-SNAPSHOT.jar importtsv
> > > -Dimporttsv.columns=HBASE_ROW_KEY,d:c1,d:c2
> > > -Dimporttsv.bulk.output=hdfs:///tmp/hfiles simple
> hdfs:///tmp/simple.tsv
> > >
> > > ...
> > >
> > > Error: java.lang.ClassNotFoundException:
> > >
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$Mutate$MutateType
> > > at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> > >  at java.security.AccessController.doPrivileged(Native Method)
> > > at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> > >  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> > > at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
> > >  at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:87)
> > >  at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.MutationSerialization$MutationSerializer.serialize(MutationSerialization.java:70)
> > > at
> > >
> >
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.collect(MapTask.java:1069)
> > >  at
> > >
> >
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.write(MapTask.java:691)
> > > at
> > >
> > >
> >
> org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
> > >  at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:151)
> > > at
> > >
> > >
> >
> org.apache.hadoop.hbase.mapreduce.TsvImporterMapper.map(TsvImporterMapper.java:37)
> > >  at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
> > > at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
> > >  at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
> > > at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
> > >  at java.security.AccessController.doPrivileged(Native Method)
> > > at javax.security.auth.Subject.doAs(Subject.java:396)
> > >  at
> > >
> > >
> >
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121)
> > > at org.apache.hadoop.mapred.Child.main(Child.java:249)
> > >
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB