Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - Example mvn cmd line to build snappy native lib?


+
Tim Broberg 2011-10-29, 06:52
+
Tim Broberg 2011-10-31, 18:24
+
Alejandro Abdelnur 2011-10-31, 19:45
+
Tim Broberg 2011-10-31, 19:59
+
Tim Broberg 2011-10-31, 21:56
Copy link to this message
-
Re: Example mvn cmd line to build snappy native lib?
Alejandro Abdelnur 2011-10-31, 22:34
You still need to download//build/install snappy. I asume you are doing
that.

Yes, Hadoop snappy JNI goes in the libhadoop OS

Thanks.

Alejandro

On Mon, Oct 31, 2011 at 2:56 PM, Tim Broberg <[EMAIL PROTECTED]> wrote:

> Solved - In trunk, the snappy symbols are getting linked in with the rest
> of the native stuff in libhadoop.so:
>
> [tbroberg@san-mothra trunk]$ nm -gC
> hadoop-common-project/hadoop-common/target/hadoop-common-0.24.0-SNAPSHOT/lib/libhadoop.so
> | grep -i snappy
> 0000000000003c20 T
> Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_compressBytesDirect
> 00000000000039e0 T
> Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_initIDs
> 0000000000004200 T
> Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_decompressBytesDirect
> 0000000000003fc0 T
> Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_initIDs
>
> So, this command line is sufficient:
>
> mvn install -Pdist,native -DskipTests
>
> Thanks again for answering, Alejandro.
>
>    - Tim.
>
> ________________________________________
> From: Tim Broberg [[EMAIL PROTECTED]]
> Sent: Monday, October 31, 2011 12:59 PM
> To: [EMAIL PROTECTED]
> Subject: RE: Example mvn cmd line to build snappy native lib?
>
> Thanks, Alejandro, but perhaps this is the procedure for a previous
> version?
>
> SVN trunk provides source in
> trunk/hadoop-common-project/hadoop-common/target/native/src/org/apache/hadoop/io/compress/snappy,
> but there is no config script or makefile.
>
> There is a fragment of documentation in trunk/BUILDING.txt - "  * Use
> -Dsnappy.prefix=(/usr/local) & -Dbundle.snappy=(false) to compile
>    Snappy JNI bindings and to bundle Snappy SO files" which suggests that
> this is maven-able, and compilation / packaging succeed, but there is no
> libsnappy.so at the end of the process.
>
> I'd really like not to switch to a different branch of the code yet again,
> and surely *somebody* knows how to build snappy in the trunk...
>
>    - Tim.
>
> ________________________________________
> From: Alejandro Abdelnur [[EMAIL PROTECTED]]
> Sent: Monday, October 31, 2011 12:45 PM
> To: [EMAIL PROTECTED]; Tim Broberg
> Subject: Re: Example mvn cmd line to build snappy native lib?
>
> Tim,
>
>  You have to download it snappy from source tarball, run './configure' and
> then 'make install'
>
> Thanks.
>
> Alejandro
>
> On Mon, Oct 31, 2011 at 11:24 AM, Tim Broberg <[EMAIL PROTECTED]> wrote:
>
> > bump
> >
> > Does anybody know how to build the snappy native library?
> >
> >     - Tim.
> >
> >
> > ________________________________
> > From: Tim Broberg <[EMAIL PROTECTED]>
> > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> > Sent: Friday, October 28, 2011 11:52 PM
> > Subject: Example mvn cmd line to build snappy native lib?
> >
> > I'm trying to build the trunk from hadoop SVN including all the native
> > libraries.
> >
> > The BUILDING.txt file has the following documentation on building the
> > native libraries:
> >
> > "
> >  Build options:
> >
> >   * Use -Pnative to compile/bundle native code
> >   * Use -Dsnappy.prefix=(/usr/local) & -Dbundle.snappy=(false) to compile
> >     Snappy JNI bindings and to bundle Snappy SO files
> > "
> >
> > I tried the following, but never seem to get a libsnappy.so anywhere:
> >
> > mvn compile -Pnative
> > mvn compile -Pnative -Dsnappy.prefix=/usr/local
> >
> > mvn compile -Pnative -Dsnappy.prefix=/usr/local -Dbundle.snappy=true
> >
> > mvn compile -Pnative -Dsnappy.prefix=/usr/local -Dbundle.snappy=false
> >
> > mvn compile -Pnative -Dsnappy.prefix=~tbroberg -Dbundle.snappy=true
> >
> >
> > If I stick a #error statement in SnappyCompressor.c, I do see the first
> of
> > these fail, so it's doing *something* with snappy.
> >
> > Can anyone provide an example command line to build this?
> >
> > Thanks,
> >     - Tim.
>
> The information and any attached documents contained in this message
> may be confidential and/or legally privileged.  The message is
+
Tim Broberg 2011-10-31, 23:33
+
Alejandro Abdelnur 2011-10-31, 23:39