Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hive, mail # user - NoClassDefFoundError: org/apache/hadoop/mapreduce/util/HostUtil


Copy link to this message
-
Re: NoClassDefFoundError: org/apache/hadoop/mapreduce/util/HostUtil
kulkarni.swarnim@...) 2013-02-07, 16:30
One reason I know for this error is not setting up HADOOP_HOME. It is right
to not set this variable since it was deprecated and replaced with
HADOOP_PREFIX and HADOOP_MAPRED_HOME. However, it seems like hive still has
some haunting references to HADOOP_HOME causing this error, specially after
the query has failed and it attempts to grab debugging info from the
tasktracker URL.

As far as your job is concerned, the failure has nothing to do with this
error. Check your tasktrackers logs to know what blew up.
On Thu, Feb 7, 2013 at 5:33 AM, Viral Bajaria <[EMAIL PROTECTED]>wrote:

> Are you seeing this after a few of the jobs have finished or on the first
> stage itself ? Also is this error on all boxes or just a few ? You can
> check MR logs and see which box or boxes are the culprits and debug from
> there.
>
> Viral
> ------------------------------
> From: Krishna Rao
> Sent: 2/7/2013 2:46 AM
> To: [EMAIL PROTECTED]
> Subject: NoClassDefFoundError: org/apache/hadoop/mapreduce/util/HostUtil
>
> Hi all,
>
> I'm occasionally getting the following error, usually after running an
> expensive Hive query (creating 20 or so MR jobs):
>
> ***
> Error during job, obtaining debugging information...
> Examining task ID: task_201301291405_1640_r_000001 (and more) from job
> job_201301291405_1640
> Exception in thread "Thread-29" java.lang.NoClassDefFoundError:
> org/apache/hadoop/mapreduce/util/HostUtil
>         at
> org.apache.hadoop.hive.shims.Hadoop23Shims.getTaskAttemptLogUrl(Hadoop23Shims.java:51)
>         at
> org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.getTaskInfos(JobDebugger.java:186)
>         at
> org.apache.hadoop.hive.ql.exec.JobDebugger$TaskInfoGrabber.run(JobDebugger.java:142)
>         at java.lang.Thread.run(Thread.java:662)
> Caused by: java.lang.ClassNotFoundException:
> org.apache.hadoop.mapreduce.util.HostUtil
>         at java.net.URLClassLoader$1.run(URLClassLoader.java:202)
>         at java.security.AccessController.doPrivileged(Native Method)
>         at java.net.URLClassLoader.findClass(URLClassLoader.java:190)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:306)
>         at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:301)
>         at java.lang.ClassLoader.loadClass(ClassLoader.java:247)
>         ... 4 more
> CmdRunner::runCmd: Error running cmd in script, error: FAILED: Execution
> Error, return code 2 from org.apache.hadoop.hive.ql.exec.MapRedTask
> ***
>
> Any ideas on what's causing it?
> How can I find out more info on this error?
>
> Cheers,
>
> Krishna
>

--
Swarnim