Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # dev >> Example mvn cmd line to build snappy native lib?


+
Tim Broberg 2011-10-29, 06:52
+
Tim Broberg 2011-10-31, 18:24
+
Alejandro Abdelnur 2011-10-31, 19:45
+
Tim Broberg 2011-10-31, 19:59
+
Tim Broberg 2011-10-31, 21:56
+
Alejandro Abdelnur 2011-10-31, 22:34
+
Tim Broberg 2011-10-31, 23:33
Copy link to this message
-
Re: Example mvn cmd line to build snappy native lib?
correct

On Mon, Oct 31, 2011 at 4:33 PM, Tim Broberg <[EMAIL PROTECTED]> wrote:

> Download from google code?
>
> ________________________________________
> From: Alejandro Abdelnur [[EMAIL PROTECTED]]
> Sent: Monday, October 31, 2011 3:34 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Example mvn cmd line to build snappy native lib?
>
> You still need to download//build/install snappy. I asume you are doing
> that.
>
> Yes, Hadoop snappy JNI goes in the libhadoop OS
>
> Thanks.
>
> Alejandro
>
> On Mon, Oct 31, 2011 at 2:56 PM, Tim Broberg <[EMAIL PROTECTED]> wrote:
>
> > Solved - In trunk, the snappy symbols are getting linked in with the rest
> > of the native stuff in libhadoop.so:
> >
> > [tbroberg@san-mothra trunk]$ nm -gC
> >
> hadoop-common-project/hadoop-common/target/hadoop-common-0.24.0-SNAPSHOT/lib/libhadoop.so
> > | grep -i snappy
> > 0000000000003c20 T
> >
> Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_compressBytesDirect
> > 00000000000039e0 T
> > Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_initIDs
> > 0000000000004200 T
> >
> Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_decompressBytesDirect
> > 0000000000003fc0 T
> > Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_initIDs
> >
> > So, this command line is sufficient:
> >
> > mvn install -Pdist,native -DskipTests
> >
> > Thanks again for answering, Alejandro.
> >
> >    - Tim.
> >
> > ________________________________________
> > From: Tim Broberg [[EMAIL PROTECTED]]
> > Sent: Monday, October 31, 2011 12:59 PM
> > To: [EMAIL PROTECTED]
> > Subject: RE: Example mvn cmd line to build snappy native lib?
> >
> > Thanks, Alejandro, but perhaps this is the procedure for a previous
> > version?
> >
> > SVN trunk provides source in
> >
> trunk/hadoop-common-project/hadoop-common/target/native/src/org/apache/hadoop/io/compress/snappy,
> > but there is no config script or makefile.
> >
> > There is a fragment of documentation in trunk/BUILDING.txt - "  * Use
> > -Dsnappy.prefix=(/usr/local) & -Dbundle.snappy=(false) to compile
> >    Snappy JNI bindings and to bundle Snappy SO files" which suggests that
> > this is maven-able, and compilation / packaging succeed, but there is no
> > libsnappy.so at the end of the process.
> >
> > I'd really like not to switch to a different branch of the code yet
> again,
> > and surely *somebody* knows how to build snappy in the trunk...
> >
> >    - Tim.
> >
> > ________________________________________
> > From: Alejandro Abdelnur [[EMAIL PROTECTED]]
> > Sent: Monday, October 31, 2011 12:45 PM
> > To: [EMAIL PROTECTED]; Tim Broberg
> > Subject: Re: Example mvn cmd line to build snappy native lib?
> >
> > Tim,
> >
> >  You have to download it snappy from source tarball, run './configure'
> and
> > then 'make install'
> >
> > Thanks.
> >
> > Alejandro
> >
> > On Mon, Oct 31, 2011 at 11:24 AM, Tim Broberg <[EMAIL PROTECTED]>
> wrote:
> >
> > > bump
> > >
> > > Does anybody know how to build the snappy native library?
> > >
> > >     - Tim.
> > >
> > >
> > > ________________________________
> > > From: Tim Broberg <[EMAIL PROTECTED]>
> > > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> > > Sent: Friday, October 28, 2011 11:52 PM
> > > Subject: Example mvn cmd line to build snappy native lib?
> > >
> > > I'm trying to build the trunk from hadoop SVN including all the native
> > > libraries.
> > >
> > > The BUILDING.txt file has the following documentation on building the
> > > native libraries:
> > >
> > > "
> > >  Build options:
> > >
> > >   * Use -Pnative to compile/bundle native code
> > >   * Use -Dsnappy.prefix=(/usr/local) & -Dbundle.snappy=(false) to
> compile
> > >     Snappy JNI bindings and to bundle Snappy SO files
> > > "
> > >
> > > I tried the following, but never seem to get a libsnappy.so anywhere:
> > >
> > > mvn compile -Pnative
> > > mvn compile -Pnative -Dsnappy.prefix=/usr/local
> > >
> > > mvn compile -Pnative -Dsnappy.prefix=/usr/local -Dbundle.snappy=true