Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive >> mail # user >> Getting CNF during map reduce execution


Copy link to this message
-
Re: Getting CNF during map reduce execution
Hi,

Sorry I forgot to mention the versions. I am running Hive 0.8.1 and Hadoop
0.20.203.0. Shashwat's suggestion worked :). (I dropped hive-exec to hadoop
lib). Thanks. Anyway I am curious how this happened since with hive running
standalone (i mean not inside OSGi) this issue didn't come up. May be
hive-exec not getting uploaded to hdfs properly during hadoop job
submission?

Regards
Buddhika
On Fri, Apr 6, 2012 at 1:21 AM, shashwat shriparv <[EMAIL PROTECTED]
> wrote:

> Your hive jars in hadoop and restart the hadoop then try.
>
>
> On Fri, Apr 6, 2012 at 12:49 AM, Carl Steinbach <[EMAIL PROTECTED]> wrote:
>
>> HI Buddhika,
>>
>> Which version of Hive are you running?
>>
>> Thanks.
>>
>> Carl
>>
>> On Thu, Apr 5, 2012 at 10:12 AM, buddhika chamith <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Hi All,
>>>
>>> I am trying to use hive inside an OSGi enviornment. So far I faced
>>> several OSGi issues in hive and some issues related to JDO implementation
>>> used (datanucleaus) under the OSGi enviornemnt and was able to overcome
>>> them. However when I run a query involving a map reduce operation I am
>>> getting a ClassNotFound exception from hadoop side. (Hadoop is running in
>>> pseudo distributed mode in the local machine).
>>>
>>> java.lang.RuntimeException: java.lang.RuntimeException: java.lang.ClassNotFoundException: org.apache.hadoop.hive.shims.HadoopShimsSecure$NullOutputCommitter
>>> at org.apache.hadoop.conf.Configuration.getClass(Configuration.java:898)
>>> at org.apache.hadoop.mapred.JobConf.getOutputCommitter(JobConf.java:613)
>>> at org.apache.hadoop.mapred.Task.initialize(Task.java:492)
>>> at org.apache.hadoop.mapred.MapTask.run(MapTask.java:352)
>>>
>>>
>>> I think this is probably due to some jar/ resource not being bundled
>>> properly in to the deployed archive to Hadoop. Would some one be able to
>>> explain a bit of how the job submission part from Hive to Hadoop works and
>>> any possible reason for this behaviour? Any pointers to look in to the code
>>> is also appreciated.
>>>
>>> Regards
>>> Buddhika
>>>
>>
>>
>
>
> --
>
>
> ∞
> Shashwat Shriparv
>
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB