Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - libjars options


Copy link to this message
-
Re: libjars options
Ted Yu 2011-01-11, 18:23
Refer to Alex Kozlov's answer on 12/11/10

On Tue, Jan 11, 2011 at 10:10 AM, C.V.Krishnakumar Iyer
<[EMAIL PROTECTED]>wrote:

> Hi,
>
> Could anyone please guide me as to how to use the -libjars option in HDFS?
>
> I have added the necessary jar file (the hbase jar - to be precise)  to the
> classpath of the node where I am starting the job.
>
> The following is the format that i am invoking:
> bin/hadoop jar <Our Jar> <MainClass> -libjars <Dependent jars (separated by
> Commas)> <Arguments to our main class>
>
> bin/hadoop jar /Users/hdp/cvk/myjob.jar  mr2.mr2a.MR2ADriver -libjars
> /Users/hdp/hadoop/lib/hbase-0.20.6.jar inputmr2a  outputmr2a
>
> Despite this,  I find that I get the java.lang.ClassNotFoundException
> error! :(
> java.lang.RuntimeException: java.lang.RuntimeException:
> java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.io.ImmutableBytesWritable
>        at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:841)
>        at
> org.apache.hadoop.mapred.JobConf.getMapOutputValueClass(JobConf.java:551)
>        at
> org.apache.hadoop.mapred.MapTask$MapOutputBuffer.(MapTask.java:793)
>        at
> org.apache.hadoop.mapred.MapTask$NewOutputCollector.(MapTask.java:524)
>        at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:613)
>        at org.apache.hadoop.mapred.MapTask.run(MapTask.java:305)
>        at org.apache.hadoop.mapred.Child.main(Child.java:170)
> Caused by: java.lang.RuntimeException: java.lang.ClassNotFoundException:
> org.apache.hadoop.hbase.io.ImmutableBytesWritable
>        at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:809)
>        at
> org.apache.hadoop.conf.Configuration.getClass(Configuration.java:833)
>
>  The strange thing is that there is another MR job I have  that runs
> perfectly with the libjars option! Could anybody tell me what I am doing
> wrong? One more thing - not sure if it is relevant : I am using the new
> Hadoop MapReduce API.
>
> Thanks in advance!
>
> Regards,
> Krishnakumar.