Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Getting CNF during map reduce execution


Copy link to this message
-
Re: Getting CNF during map reduce execution
Hi,

Sorry I forgot to mention the versions. I am running Hive 0.8.1 and Hadoop
0.20.203.0. Shashwat's suggestion worked :). (I dropped hive-exec to hadoop
lib). Thanks. Anyway I am curious how this happened since with hive running
standalone (i mean not inside OSGi) this issue didn't come up. May be
hive-exec not getting uploaded to hdfs properly during hadoop job
submission?

Regards
Buddhika
On Fri, Apr 6, 2012 at 1:21 AM, shashwat shriparv <[EMAIL PROTECTED]
> wrote:

> Your hive jars in hadoop and restart the hadoop then try.
>
>
> On Fri, Apr 6, 2012 at 12:49 AM, Carl Steinbach <[EMAIL PROTECTED]> wrote:
>
>> HI Buddhika,
>>
>> Which version of Hive are you running?
>>
>> Thanks.
>>
>> Carl
>>
>> On Thu, Apr 5, 2012 at 10:12 AM, buddhika chamith <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi All,
>>>
>>> I am trying to use hive inside an OSGi enviornment. So far I faced
>>> several OSGi issues in hive and some issues related to JDO implementation
>>> used (datanucleaus) under the OSGi enviornemnt and was able to overcome
>>> them. However when I run a query involving a map reduce operation I am
>>> getting a ClassNotFound exception from hadoop side. (Hadoop is running in
>>> pseudo distributed mode in the local machine).
>>>
>>> java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.shims.HadoopShimsSecure$NullOutputCommitter
>>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:898)
>>> at org.apache.hadoop.mapred.JobConf.getOutputCommitter(JobConf.java:613)
>>> at org.apache.hadoop.mapred.Task.initialize(Task.java:492)
>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:352)
>>>
>>>
>>> I think this is probably due to some jar/ resource not being bundled
>>> properly in to the deployed archive to Hadoop. Would some one be able to
>>> explain a bit of how the job submission part from Hive to Hadoop works and
>>> any possible reason for this behaviour? Any pointers to look in to the code
>>> is also appreciated.
>>>
>>> Regards
>>> Buddhika
>>>
>>
>>
>
>
> --
>
>
> ∞
> Shashwat Shriparv
>
>
>