Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> Re: Run jar built with new apis on a CDH3 cluster


Copy link to this message
-
Re: Run jar built with new apis on a CDH3 cluster
It isn't possible to do this since 0.21+ broke MR binary compatibility
aside of some other minor API changes. You do not need to rewrite your
application/job in most cases, but you do have to recompile it against the
target cluster's libraries to get a binary compatible jar you can run.
On Thu, Mar 14, 2013 at 6:42 PM, Zheyi RONG <[EMAIL PROTECTED]> wrote:

> Hi,
>
> Is it possible to run a jar built with hadoop new apis (hadoop 0.22.0) on
> a cluster deployed with CDH 3 (hadoop 0.20.2) ?
> The jar uses some hadoop Api (e.g.
> org.apache.hadoop.mapreduce.lib.chain.ChainMapper) that hadoop 0.20.2 does
> not have.
>
> I ran it, but hadoop complains:
> Exception in thread "main" java.lang.IncompatibleClassChangeError:
> Implementing class
> at java.lang.ClassLoader.defineClass1(Native Method)
>  at java.lang.ClassLoader.defineClassCond(ClassLoader.java:631)
> at java.lang.ClassLoader.defineClass(ClassLoader.java:615)
>  at
> java.security.SecureClassLoader.defineClass(SecureClassLoader.java:141)
> at java.net.URLClassLoader.defineClass(URLClassLoader.java:283)
>  at java.net.URLClassLoader.access$000(URLClassLoader.java:58)
> at java.net.URLClassLoader$1.run(URLClassLoader.java:197)
>  at java.security.AccessController.doPrivileged(Native Method)
> at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>  at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
> at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>  at
> org.apache.hadoop.mapreduce.lib.chain.ChainMapper.addMapper(ChainMapper.java:126)
> at driver.ChainDriver.run(ChainDriver.java:146)
>  at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:65)
> at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:79)
>  at driver.ChainDriver.main(ChainDriver.java:173)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
>  at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
>  at java.lang.reflect.Method.invoke(Method.java:597)
> at org.apache.hadoop.util.RunJar.main(RunJar.java:197)
>
>
> Is there any workaround? I really do not want to rewrite my project...
>
> Thank you very much.
>
> Regards,
>
> Zheyi.
>

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB