Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Setting ClassLoader in HBaseConfiguration


Copy link to this message
-
Re: Setting ClassLoader in HBaseConfiguration
Amit Sela 2013-04-22, 16:04
Well, not really... utilizing the method doesn't really work for me - Since
the constructors in HBaseConfiguration are deprecated and the way to use it
is by calling the static create methods, I can't really use the
setClassLoader method from Configuration. In addition to that, I need to
set the ClassLoader before create executes the new Configuration and
addResources because all configurations to set are in the bundle context.

Even when implementing my own HBaseConfiguration (loading the class loader
before new Configuration() and addHBaseResources()), I get an exception
when trying to create a new HTableInterface because HConnectionManager has
a static block that calls HBaseConfiguration.create() (line: 148) and the
configuration returned doesn't have the right class loader...

The only way I found so far (without changing HBase code) is by
calling: Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
before executing my code.

Hadoop uses Configuration when submitting jobs for example, and it has
clone constructor that can pass the class loader (and more), but from what
I saw the HBase framework uses create() and even if using
create(Configuration) the merge method merges only <String, String>
properties map.

Any ideas ? Should I stay with my bypass ? I saw a TODO to replace the
constructor with a private one, and not extend Configuration, so maybe also
allow setting of ClassLoader and use clone constructors in the framework
(for instance HConnectionManager will use a cloned configuration when
calling new HTable()).

Thanks.

On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Since HBaseConfiguration extends Configuration, can you utilize this method
> from Configuration ?
>
>   public void setClassLoader(ClassLoader classLoader) {
> Thanks
>
> On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
> >
> > I'm trying to run an HBase client from an OSGI environment and for that I
> > need to set the Configuration classLoader.
> > In Configuration (Hadoop) itself, there is a method for that but since
> > HBaseConfiguration.create() is static the only solution I found was:
> >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > before calling HBaseConfiguration.create() .
> >
> > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> >
> > Thanks,
> >
> > Amit.
> >
>

On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Since HBaseConfiguration extends Configuration, can you utilize this method
> from Configuration ?
>
>   public void setClassLoader(ClassLoader classLoader) {
> Thanks
>
> On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
> >
> > I'm trying to run an HBase client from an OSGI environment and for that I
> > need to set the Configuration classLoader.
> > In Configuration (Hadoop) itself, there is a method for that but since
> > HBaseConfiguration.create() is static the only solution I found was:
> >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > before calling HBaseConfiguration.create() .
> >
> > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> >
> > Thanks,
> >
> > Amit.
> >
>

On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <[EMAIL PROTECTED]> wrote:

> Since HBaseConfiguration extends Configuration, can you utilize this method
> from Configuration ?
>
>   public void setClassLoader(ClassLoader classLoader) {
> Thanks
>
> On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
>
> > Hi all,
> >
> > I'm trying to run an HBase client from an OSGI environment and for that I
> > need to set the Configuration classLoader.
> > In Configuration (Hadoop) itself, there is a method for that but since
> > HBaseConfiguration.create() is static the only solution I found was:
> >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());