Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce, mail # user - Bug in LocalJobRunner?


Copy link to this message
-
Re: Bug in LocalJobRunner?
Alex Baranau 2013-03-22, 21:57
Hi Harsh J,

Thanx for taking a look. I created
https://issues.apache.org/jira/browse/MAPREDUCE-5097 and attached patch. I
also provided (ugly, sorry) example of how to get the error.

Alex Baranau

On Thu, Mar 21, 2013 at 5:58 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hi Alex,
>
> This seems to make sense. Could you post your patch with a possible
> (even if detached) test case?
>
> On Thu, Mar 21, 2013 at 6:14 AM, Alex Baranau <[EMAIL PROTECTED]>
> wrote:
> > Hi!
> >
> > In a test I tried to run a job which is in separate jar with
> > MiniMRYarnCluster and got ClassNotFoundException for the Mapper class
> that
> > is in that jar.
> >
> > I went a bit deeper and found possible bug in LocalJobRunner:
> >
> > currently:
> >
> >       this.job = new JobConf(localJobFile);
> >
> >       // Job (the current object) is a Thread, so we wrap its class
> loader.
> >       if (localDistributedCacheManager.hasLocalClasspaths()) {
> >
> setContextClassLoader(localDistributedCacheManager.makeClassLoader(
> >                 getContextClassLoader()));
> >       }
> >
> > should be:
> >
> >       this.job = new JobConf(localJobFile);
> >       // Job (the current object) is a Thread, so we wrap its class
> loader.
> >       if (localDistributedCacheManager.hasLocalClasspaths()) {
> >         ClassLoader classLoader > > localDistributedCacheManager.makeClassLoader(getContextClassLoader());
> >         setContextClassLoader(classLoader);
> >         this.job.setClassLoader(classLoader);
> >       }
> >
> > I.e. we need to set classloader for job configuration so that it can load
> > classes from the jar.
> >
> > If the above makes sense I will file JIRA with patch, otherwise, what am
> I
> > missing?
> >
> >
> > Thank you,
> > Alex Baranau
>
>
>
> --
> Harsh J
>