Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # dev >> loading Hadoop native libraries in HBase unit tests


+
Mikhail Bautin 2012-02-11, 03:22
+
Ted Yu 2012-02-11, 03:26
+
Jesse Yates 2012-02-11, 03:30
+
Stack 2012-02-11, 04:52
+
Mikhail Bautin 2012-02-12, 20:36
+
Roman Shaposhnik 2012-02-13, 06:02
Copy link to this message
-
Re: loading Hadoop native libraries in HBase unit tests
Then how about solving the issue for the most common case (the default
version of Hadoop)? We can import the default version of libhadoop.so into
the HBase codebase and load it in tests, as I mentioned. This can be
considered a hack but will definitely increase the test coverage.

Thanks,
--Mikhail

On Sun, Feb 12, 2012 at 10:02 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:

> On Sun, Feb 12, 2012 at 12:36 PM, Mikhail Bautin
> <[EMAIL PROTECTED]> wrote:
> > One difficulty with simply importing libhadoop.so into HBase codebase is
> > that the dynamic library is probably a bit different for different
> versions
> > of Hadoop. Is there a way to pull the .so file from Maven for the
> > configured Hadoop version? Ideally this should be done in a
> > platform-independent way, too, but making it work on Linux would be the
> > first step.
>
> I'm pretty sure it'll be next to impossible to do that reliably. There used
> to be a time when folks wanted to use Maven for the native artifacts,
> but to the best of my knowledge that has been deemed to be
> "not such a good idea" (tm) after all.
>
> In fact if you look at Maven plugins dealing with native side of things
> pretty
> much most of them seem to be abandoned at this point.
>
> Personally, working with Maven has made me realize that the only
> reliable way to deal with external dependencies on native artifact
> is to depend on source artifact and always do the compilation
> during your own build. You mileage may, of course, vary.
>
> Thanks,
> Roman.
>
+
Roman Shaposhnik 2012-02-13, 18:52
+
Todd Lipcon 2012-02-13, 19:15
+
Mikhail Bautin 2012-02-13, 19:21
+
Todd Lipcon 2012-02-13, 19:41
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB