|
|
-
Re: Issue with running hadoop program using eclipseHemanth Yamijala 2013-02-01, 04:12
Previously, I have resolved this error by building a jar and then using the
API job.setJarByClass(<driver-class>.class). Can you please try that once ? On Thu, Jan 31, 2013 at 6:40 PM, Vikas Jadhav <[EMAIL PROTECTED]>wrote: > Hi I know it class not found error > but I have Map and reduce Class as part of Driver class > So what is problem ? > > I want to ask whether it is compulsory to have jar of any Hadoop Program > Because it say "No job jar file set " See JobConf(Class) or > JobConf#setJar(String). > > I am running my program using eclipse(windows machine) and Hadoop(llinux > machine) remotely. > > > > On Thu, Jan 31, 2013 at 12:40 PM, Mohammad Tariq <[EMAIL PROTECTED]>wrote: > >> Hello Vikas, >> >> It clearly shows that the class can not be found. For >> debugging, you can write your MR job as a standalone java program and debug >> it. It works. And if you want to just debug your mapper / reducer logic, >> you should look into using MRUnit. There is a good write-up<http://blog.cloudera.com/blog/2009/07/debugging-mapreduce-programs-with-mrunit/>at Cloudera's blog section which talks about it in detail. >> >> HTH >> >> Warm Regards, >> Tariq >> https://mtariq.jux.com/ >> cloudfront.blogspot.com >> >> >> On Thu, Jan 31, 2013 at 11:56 AM, Vikas Jadhav <[EMAIL PROTECTED]>wrote: >> >>> Hi >>> I have one windows machine and one linux machine >>> my eclipse is on winowds machine >>> and hadoop running on single linux machine >>> I am trying to run wordcount program from eclipse(on windows machine) to >>> Hadoop(on linux machine) >>> I getting following error >>> >>> 3/01/31 11:48:14 WARN mapred.JobClient: Use GenericOptionsParser for >>> parsing the arguments. Applications should implement Tool for the same. >>> 13/01/31 11:48:15 WARN mapred.JobClient: No job jar file set. User >>> classes may not be found. See JobConf(Class) or JobConf#setJar(String). >>> 13/01/31 11:48:16 INFO input.FileInputFormat: Total input paths to >>> process : 1 >>> 13/01/31 11:48:16 WARN util.NativeCodeLoader: Unable to load >>> native-hadoop library for your platform... using builtin-java classes where >>> applicable >>> 13/01/31 11:48:16 WARN snappy.LoadSnappy: Snappy native library not >>> loaded >>> 13/01/31 11:48:26 INFO mapred.JobClient: Running job: >>> job_201301300613_0029 >>> 13/01/31 11:48:27 INFO mapred.JobClient: map 0% reduce 0% >>> 13/01/31 11:48:40 INFO mapred.JobClient: Task Id : >>> attempt_201301300613_0029_m_000000_0, Status : FAILED >>> java.lang.RuntimeException: java.lang.ClassNotFoundException: >>> WordCount$MapClass >>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:867) >>> at >>> org.apache.hadoop.mapreduce.JobContext.getMapperClass(JobContext.java:199) >>> at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:719) >>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370) >>> at org.apache.hadoop.mapred.Child$4.run(Child.java:255) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at javax.security.auth.Subject.doAs(Subject.java:396) >>> at >>> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1121) >>> at org.apache.hadoop.mapred.Child.main(Child.java:249) >>> Caused by: java.lang.ClassNotFoundException: WordCount$MapClass >>> at java.net.URLClassLoader$1.run(URLClassLoader.java:202) >>> at java.security.AccessController.doPrivileged(Native Method) >>> at java.net.URLClassLoader.findClass(URLClassLoader.java:190) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:306) >>> at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301) >>> at java.lang.ClassLoader.loadClass(ClassLoader.java:247) >>> at java.lang.Class.forName0(Native Method) >>> at java.lang.Class.forName(Class.java:247) >>> at >>> org.apache.hadoop.conf.Configuration.getClassByName(Configuration.java:820) >>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:865) >>> ... 8 more >>> >>> >>> >>> >>> Also I want to know how to debug Hadoop Program using eclipse. +
Mohammad Tariq 2013-02-01, 07:53
|