Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> loading Hadoop native libraries in HBase unit tests


+
Mikhail Bautin 2012-02-11, 03:22
+
Ted Yu 2012-02-11, 03:26
+
Jesse Yates 2012-02-11, 03:30
+
Stack 2012-02-11, 04:52
+
Mikhail Bautin 2012-02-12, 20:36
+
Roman Shaposhnik 2012-02-13, 06:02
+
Mikhail Bautin 2012-02-13, 09:58
+
Roman Shaposhnik 2012-02-13, 18:52
+
Todd Lipcon 2012-02-13, 19:15
Copy link to this message
-
Re: loading Hadoop native libraries in HBase unit tests
Would the following work as a complete solution for any platform? We can
make this conditional on a new Maven profile.

   - Download the sources of the Hadoop version being used
   - Run "ant compile-native"
   - Add the directory of libhadoop.so to java.library.path in the test JVM
   options

Thanks,
--Mikhail

On Mon, Feb 13, 2012 at 11:15 AM, Todd Lipcon <[EMAIL PROTECTED]> wrote:

> Also keep in mind it's not just the hadoop version, but also the glibc
> version and host architecture. We'd have to publish built binaries for
> all combinations of architecture*hadoopVersion*glibcVersion
>
> Maybe we should just get a copy of _one_ of these versions on the
> hudson build boxes, and have a new hudson job which runs whichever
> tests depend on the native code there?
>
> -Todd
>
> On Mon, Feb 13, 2012 at 10:52 AM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> > On Mon, Feb 13, 2012 at 1:58 AM, Mikhail Bautin
> > <[EMAIL PROTECTED]> wrote:
> >> Then how about solving the issue for the most common case (the default
> >> version of Hadoop)? We can import the default version of libhadoop.so
> into
> >> the HBase codebase and load it in tests, as I mentioned. This can be
> >> considered a hack but will definitely increase the test coverage.
> >
> > You're not proposing importing a native binary into a source tree, are
> you?
> > That won't be very reliable at all.
> >
> > We can probably come up with a # of workaroudns here, but at the end
> > of the day, unless you recompiled the native bits here and now, chances
> > are they won't be compatible with the OS you happen to be on.
> >
> > Thanks,
> > Roman.
>
>
>
> --
> Todd Lipcon
> Software Engineer, Cloudera
>
+
Todd Lipcon 2012-02-13, 19:41