Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hive, mail # user - Getting CNF during map reduce execution


Copy link to this message
-
Getting CNF during map reduce execution
buddhika chamith 2012-04-05, 17:12
Hi All,

I am trying to use hive inside an OSGi enviornment. So far I faced several
OSGi issues in hive and some issues related to JDO implementation used
(datanucleaus) under the OSGi enviornemnt and was able to overcome them.
However when I run a query involving a map reduce operation I am getting a
ClassNotFound exception from hadoop side. (Hadoop is running in pseudo
distributed mode in the local machine).

java.lang.RuntimeException: java.lang.RuntimeException:
java.lang.ClassNotFoundException:
org.apache.hadoop.hive.shims.HadoopShimsSecure$NullOutputCommitter
at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:898)
at org.apache.hadoop.mapred.JobConf.getOutputCommitter(JobConf.java:613)
at org.apache.hadoop.mapred.Task.initialize(Task.java:492)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:352)
I think this is probably due to some jar/ resource not being bundled
properly in to the deployed archive to Hadoop. Would some one be able to
explain a bit of how the job submission part from Hive to Hadoop works and
any possible reason for this behaviour? Any pointers to look in to the code
is also appreciated.

Regards
Buddhika
+
Carl Steinbach 2012-04-05, 19:19
+
shashwat shriparv 2012-04-05, 19:21
+
buddhika chamith 2012-04-09, 08:47