Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How to install Snappy?


Copy link to this message
-
Re: How to install Snappy?
There is a compression test JIRA right now.  What are you seeing?

On Mon, Dec 3, 2012 at 8:47 AM, Jean-Marc Spaggiari <[EMAIL PROTECTED]
> wrote:

> Ok....
>
> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2 helped
> me and my test program is now working. I'm able to load both
> libraries. Fine.
>
> But the CompressionTest is still not working.
>
> What is very strange is that:
> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
> available
> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not loaded
>
> It's available, but not loaded.
>
> But from the code:
>   static {
>     try {
>       System.loadLibrary("snappy");
>       System.loadLibrary("hadoopsnappy");
>       LOG.warn("Snappy native library is available");
>       AVAILABLE = true;
>     } catch (UnsatisfiedLinkError ex) {
>       //NOP
>     }
>     LOADED = AVAILABLE;
>     if (LOADED) {
>       LOG.info("Snappy native library loaded");
>     } else {
>       LOG.warn("Snappy native library not loaded");
>     }
>   }
> If "Snappy native library is available" is displayed, that mean
> AVAILABLE = true... And if AVAILABLE = true, then LOADED is set to
> true and Snappy native library loaded must be displayed... But it's
> not... How is this possible?
>
> I have not expected Snappy installation to be such a challenge...
>
> I will continue to dig and summarize the steps when I will be done (If
> I'm able to finish...)
>
> JM
>
> 2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
> > Thanks all for your replies.
> >
> > So, to reply to all in one.
> >
> > I'm not using CD3. I'm using Hadoop  1.0.3 and HBase 0.94.2 directly
> > from the JARs.
> >
> > Here are all the places where I have put the lib:
> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so
> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so.1
> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so.1.1.3
> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so
> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so.1
> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so.1.1.3
> > /home/hbase/hbase-0.94.2/lib/native/libsnappy.so
> > /home/hbase/hbase-0.94.2/lib/native/libsnappy.so.1
> > /home/hbase/hbase-0.94.2/lib/native/libsnappy.so.1.1.3
> > /home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so
> > /home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so.1
> > /home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so.1.1.3
> > /home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so
> > /home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so.1
> > /home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so.1.1.3
> > /lib/x86_64-linux-gnu/libsnappy.so
> > /usr/lib/libsnappy.so
> > /usr/lib/libsnappy.so.1
> > /usr/lib/libsnappy.so.1.1.3
> > /usr/local/lib/libsnappy.so
> > /usr/local/lib/libsnappy.so.1
> > /usr/local/lib/libsnappy.so.1.1.3
> >
> > I tried to add this on my hbase-env.xml:
> > export
> > HBASE_LIBRARY_PATH=/home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64
> >
> > Before I was trying with doing export on the command line directly
> > since it seems the hbase script is taking that into consideration too.
> >
> > I have not yet put the hbase.regionserver.codecs line since I still
> > need to use my cluster until I get snappy working. On the hbase/lib
> > directory I have snappy-java-1.0.3.2.jar.
> >
> >
> > Should snappy be installed within hbase? Or should it be in hadoop?
> > I'm not sure anymore.
> >
> > But it's still not working. So I tried the small code below:
> >
> > import java.util.StringTokenizer;
> >
> > public class Test
> > {
> >   static {
> >     try {
> >       System.loadLibrary("snappy");
> >       System.loadLibrary("hadoopsnappy");
> >       System.out.println ("Snappy native library is available");
> >     } catch (UnsatisfiedLinkError ex) {
> >         ex.printStackTrace();
> >     }
> >   }

Kevin O'Dell
Customer Operations Engineer, Cloudera