Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
MapReduce >> mail # user >> Job driver and 3rd party jars


+
Barak Yaish 2013-03-07, 18:48
Copy link to this message
-
Re: Job driver and 3rd party jars
Yep, my typo, I'm using the later. I was also trying export
HADOOP_CLASSPATH_USER_FIRST =true and export HADOOP_CLASSPATH=myjar before
launching the hadoop jar, but I still getting the same exception.
I'm running hadoop 1.0.4.
On Mar 8, 2013 2:27 AM, "Harsh J" <[EMAIL PROTECTED]> wrote:

> To be precise, did you use -libjar or -libjars? The latter is the right
> option.
>
> On Fri, Mar 8, 2013 at 12:18 AM, Barak Yaish <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> > I'm able to run M/R jobs where the mapper and reducer required to use 3rd
> > party jars. I'm registering those jars in -libjar while invoking the
> hadoop
> > jar command. I'm facing a strange problem, though, when the job driver
> > itself ( extends Configured implements Tool ) required to run such code (
> > for example notify some remote service upon start and end). Is there a
> way
> > to configure classpath when submitting jobs using hadoop jar? Seems like
> > -libjar doesn't work for this case...
> >
> > Exception in thread "main" java.lang.NoClassDefFoundError:
> > com/me/context/DefaultContext
> >         at java.lang.ClassLoader.defineClass1(Native Method)
> >         at java.lang.ClassLoader.defineClassCond(ClassLoader.java:632)
> >         at java.lang.ClassLoader.defineClass(ClassLoader.java:616)
> >         at
> > java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> >         at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
> >         at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> >         at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
> >         at
> > com.peer39.bigdata.mr.pnm.PnmDataCruncher.run(PnmDataCruncher.java:50)
> >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> >         at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
> >         at com.me.mr.pnm.PnmMR.main(PnmDataCruncher.java:261)
> >         at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> >         at
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> >         at
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> >         at java.lang.reflect.Method.invoke(Method.java:597)
> >         at org.apache.hadoop.util.RunJar.main(RunJar.java:156)
> > Caused by: java.lang.ClassNotFoundException:
> com.me.context.DefaultContext
> >         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
> >         at java.security.AccessController.doPrivileged(Native Method)
> >         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:307)
> >         at java.lang.ClassLoader.loadClass(ClassLoader.java:248)
>
>
>
> --
> Harsh J
>
+
刘晓文 2013-03-08, 07:15
+
Stephen Boesch 2013-03-10, 03:28
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB