Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # user >> Bug in LocalJobRunner?


Copy link to this message
-
Re: Bug in LocalJobRunner?
Hi Harsh J,

Thanx for taking a look. I created
https://issues.apache.org/jira/browse/MAPREDUCE-5097 and attached patch. I
also provided (ugly, sorry) example of how to get the error.

Alex Baranau

On Thu, Mar 21, 2013 at 5:58 AM, Harsh J <[EMAIL PROTECTED]> wrote:

> Hi Alex,
>
> This seems to make sense. Could you post your patch with a possible
> (even if detached) test case?
>
> On Thu, Mar 21, 2013 at 6:14 AM, Alex Baranau <[EMAIL PROTECTED]>
> wrote:
> > Hi!
> >
> > In a test I tried to run a job which is in separate jar with
> > MiniMRYarnCluster and got ClassNotFoundException for the Mapper class
> that
> > is in that jar.
> >
> > I went a bit deeper and found possible bug in LocalJobRunner:
> >
> > currently:
> >
> >       this.job = new JobConf(localJobFile);
> >
> >       // Job (the current object) is a Thread, so we wrap its class
> loader.
> >       if (localDistributedCacheManager.hasLocalClasspaths()) {
> >
> setContextClassLoader(localDistributedCacheManager.makeClassLoader(
> >                 getContextClassLoader()));
> >       }
> >
> > should be:
> >
> >       this.job = new JobConf(localJobFile);
> >       // Job (the current object) is a Thread, so we wrap its class
> loader.
> >       if (localDistributedCacheManager.hasLocalClasspaths()) {
> >         ClassLoader classLoader > > localDistributedCacheManager.makeClassLoader(getContextClassLoader());
> >         setContextClassLoader(classLoader);
> >         this.job.setClassLoader(classLoader);
> >       }
> >
> > I.e. we need to set classloader for job configuration so that it can load
> > classes from the jar.
> >
> > If the above makes sense I will file JIRA with patch, otherwise, what am
> I
> > missing?
> >
> >
> > Thank you,
> > Alex Baranau
>
>
>
> --
> Harsh J
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB