Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # dev >> Example mvn cmd line to build snappy native lib?


Copy link to this message
-
Re: Example mvn cmd line to build snappy native lib?
correct

On Mon, Oct 31, 2011 at 4:33 PM, Tim Broberg <[EMAIL PROTECTED]> wrote:

> Download from google code?
>
> ________________________________________
> From: Alejandro Abdelnur [[EMAIL PROTECTED]]
> Sent: Monday, October 31, 2011 3:34 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Example mvn cmd line to build snappy native lib?
>
> You still need to download//build/install snappy. I asume you are doing
> that.
>
> Yes, Hadoop snappy JNI goes in the libhadoop OS
>
> Thanks.
>
> Alejandro
>
> On Mon, Oct 31, 2011 at 2:56 PM, Tim Broberg <[EMAIL PROTECTED]> wrote:
>
> > Solved - In trunk, the snappy symbols are getting linked in with the rest
> > of the native stuff in libhadoop.so:
> >
> > [tbroberg@san-mothra trunk]$ nm -gC
> >
> hadoop-common-project/hadoop-common/target/hadoop-common-0.24.0-SNAPSHOT/lib/libhadoop.so
> > | grep -i snappy
> > 0000000000003c20 T
> >
> Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_compressBytesDirect
> > 00000000000039e0 T
> > Java_org_apache_hadoop_io_compress_snappy_SnappyCompressor_initIDs
> > 0000000000004200 T
> >
> Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_decompressBytesDirect
> > 0000000000003fc0 T
> > Java_org_apache_hadoop_io_compress_snappy_SnappyDecompressor_initIDs
> >
> > So, this command line is sufficient:
> >
> > mvn install -Pdist,native -DskipTests
> >
> > Thanks again for answering, Alejandro.
> >
> >    - Tim.
> >
> > ________________________________________
> > From: Tim Broberg [[EMAIL PROTECTED]]
> > Sent: Monday, October 31, 2011 12:59 PM
> > To: [EMAIL PROTECTED]
> > Subject: RE: Example mvn cmd line to build snappy native lib?
> >
> > Thanks, Alejandro, but perhaps this is the procedure for a previous
> > version?
> >
> > SVN trunk provides source in
> >
> trunk/hadoop-common-project/hadoop-common/target/native/src/org/apache/hadoop/io/compress/snappy,
> > but there is no config script or makefile.
> >
> > There is a fragment of documentation in trunk/BUILDING.txt - "  * Use
> > -Dsnappy.prefix=(/usr/local) & -Dbundle.snappy=(false) to compile
> >    Snappy JNI bindings and to bundle Snappy SO files" which suggests that
> > this is maven-able, and compilation / packaging succeed, but there is no
> > libsnappy.so at the end of the process.
> >
> > I'd really like not to switch to a different branch of the code yet
> again,
> > and surely *somebody* knows how to build snappy in the trunk...
> >
> >    - Tim.
> >
> > ________________________________________
> > From: Alejandro Abdelnur [[EMAIL PROTECTED]]
> > Sent: Monday, October 31, 2011 12:45 PM
> > To: [EMAIL PROTECTED]; Tim Broberg
> > Subject: Re: Example mvn cmd line to build snappy native lib?
> >
> > Tim,
> >
> >  You have to download it snappy from source tarball, run './configure'
> and
> > then 'make install'
> >
> > Thanks.
> >
> > Alejandro
> >
> > On Mon, Oct 31, 2011 at 11:24 AM, Tim Broberg <[EMAIL PROTECTED]>
> wrote:
> >
> > > bump
> > >
> > > Does anybody know how to build the snappy native library?
> > >
> > >     - Tim.
> > >
> > >
> > > ________________________________
> > > From: Tim Broberg <[EMAIL PROTECTED]>
> > > To: "[EMAIL PROTECTED]" <[EMAIL PROTECTED]>
> > > Sent: Friday, October 28, 2011 11:52 PM
> > > Subject: Example mvn cmd line to build snappy native lib?
> > >
> > > I'm trying to build the trunk from hadoop SVN including all the native
> > > libraries.
> > >
> > > The BUILDING.txt file has the following documentation on building the
> > > native libraries:
> > >
> > > "
> > >  Build options:
> > >
> > >   * Use -Pnative to compile/bundle native code
> > >   * Use -Dsnappy.prefix=(/usr/local) & -Dbundle.snappy=(false) to
> compile
> > >     Snappy JNI bindings and to bundle Snappy SO files
> > > "
> > >
> > > I tried the following, but never seem to get a libsnappy.so anywhere:
> > >
> > > mvn compile -Pnative
> > > mvn compile -Pnative -Dsnappy.prefix=/usr/local
> > >
> > > mvn compile -Pnative -Dsnappy.prefix=/usr/local -Dbundle.snappy=true
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB