Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Setting ClassLoader in HBaseConfiguration


Copy link to this message
-
Re: Setting ClassLoader in HBaseConfiguration
Ted Yu 2013-04-22, 16:28
The static block in HConnectionManager requires some consideration if we
are to support custom classloader in HBaseConfiguration.
Suppose you have a Configuration object with custom classloader, we need to
delay instantiating HBASE_INSTANCES so that you can pass Configuration
object to HConnectionManager.

For the moment, it looks like your bypass is the best choice.

Thanks for sharing your experiences.

On Mon, Apr 22, 2013 at 9:04 AM, Amit Sela <[EMAIL PROTECTED]> wrote:

> Well, not really... utilizing the method doesn't really work for me - Since
> the constructors in HBaseConfiguration are deprecated and the way to use it
> is by calling the static create methods, I can't really use the
> setClassLoader method from Configuration. In addition to that, I need to
> set the ClassLoader before create executes the new Configuration and
> addResources because all configurations to set are in the bundle context.
>
> Even when implementing my own HBaseConfiguration (loading the class loader
> before new Configuration() and addHBaseResources()), I get an exception
> when trying to create a new HTableInterface because HConnectionManager has
> a static block that calls HBaseConfiguration.create() (line: 148) and the
> configuration returned doesn't have the right class loader...
>
> The only way I found so far (without changing HBase code) is by
> calling:
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> before executing my code.
>
> Hadoop uses Configuration when submitting jobs for example, and it has
> clone constructor that can pass the class loader (and more), but from what
> I saw the HBase framework uses create() and even if using
> create(Configuration) the merge method merges only <String, String>
> properties map.
>
> Any ideas ? Should I stay with my bypass ? I saw a TODO to replace the
> constructor with a private one, and not extend Configuration, so maybe also
> allow setting of ClassLoader and use clone constructors in the framework
> (for instance HConnectionManager will use a cloned configuration when
> calling new HTable()).
>
> Thanks.
>
>
>
> On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Since HBaseConfiguration extends Configuration, can you utilize this
> method
> > from Configuration ?
> >
> >   public void setClassLoader(ClassLoader classLoader) {
> > Thanks
> >
> > On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
> >
> > > Hi all,
> > >
> > > I'm trying to run an HBase client from an OSGI environment and for
> that I
> > > need to set the Configuration classLoader.
> > > In Configuration (Hadoop) itself, there is a method for that but since
> > > HBaseConfiguration.create() is static the only solution I found was:
> > >
> > >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > > before calling HBaseConfiguration.create() .
> > >
> > > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> > >
> > > Thanks,
> > >
> > > Amit.
> > >
> >
>
>
>
> On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <[EMAIL PROTECTED]> wrote:
>
> > Since HBaseConfiguration extends Configuration, can you utilize this
> method
> > from Configuration ?
> >
> >   public void setClassLoader(ClassLoader classLoader) {
> > Thanks
> >
> > On Sun, Apr 21, 2013 at 10:00 AM, Amit Sela <[EMAIL PROTECTED]> wrote:
> >
> > > Hi all,
> > >
> > > I'm trying to run an HBase client from an OSGI environment and for
> that I
> > > need to set the Configuration classLoader.
> > > In Configuration (Hadoop) itself, there is a method for that but since
> > > HBaseConfiguration.create() is static the only solution I found was:
> > >
> > >
> >
> Thread.currentThread().setContextClassLoader(this.getClass().getClassLoader());
> > > before calling HBaseConfiguration.create() .
> > >
> > > Should there be HBaseConfiguration.create(ClassLoader classLoader) ?
> > >
> > > Thanks,
> > >
> > > Amit.
> > >
> >
>
>
>
> On Sun, Apr 21, 2013 at 8:12 PM, Ted Yu <[EMAIL PROTECTED]> wrote: