Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> How to install Snappy?


+
Jean-Marc Spaggiari 2012-12-01, 23:52
+
Jean-Marc Spaggiari 2012-12-01, 23:57
+
Jean-Marc Spaggiari 2012-12-02, 13:25
+
surfer 2012-12-03, 07:02
+
Jean-Marc Spaggiari 2012-12-03, 13:24
Copy link to this message
-
Re: How to install Snappy?
Ok....

This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2 helped
me and my test program is now working. I'm able to load both
libraries. Fine.

But the CompressionTest is still not working.

What is very strange is that:
12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is available
12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not loaded

It's available, but not loaded.

But from the code:
  static {
    try {
      System.loadLibrary("snappy");
      System.loadLibrary("hadoopsnappy");
      LOG.warn("Snappy native library is available");
      AVAILABLE = true;
    } catch (UnsatisfiedLinkError ex) {
      //NOP
    }
    LOADED = AVAILABLE;
    if (LOADED) {
      LOG.info("Snappy native library loaded");
    } else {
      LOG.warn("Snappy native library not loaded");
    }
  }
If "Snappy native library is available" is displayed, that mean
AVAILABLE = true... And if AVAILABLE = true, then LOADED is set to
true and Snappy native library loaded must be displayed... But it's
not... How is this possible?

I have not expected Snappy installation to be such a challenge...

I will continue to dig and summarize the steps when I will be done (If
I'm able to finish...)

JM

2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
> Thanks all for your replies.
>
> So, to reply to all in one.
>
> I'm not using CD3. I'm using Hadoop  1.0.3 and HBase 0.94.2 directly
> from the JARs.
>
> Here are all the places where I have put the lib:
> /home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so
> /home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so.1
> /home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so.1.1.3
> /home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so
> /home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so.1
> /home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so.1.1.3
> /home/hbase/hbase-0.94.2/lib/native/libsnappy.so
> /home/hbase/hbase-0.94.2/lib/native/libsnappy.so.1
> /home/hbase/hbase-0.94.2/lib/native/libsnappy.so.1.1.3
> /home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so
> /home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so.1
> /home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so.1.1.3
> /home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so
> /home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so.1
> /home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so.1.1.3
> /lib/x86_64-linux-gnu/libsnappy.so
> /usr/lib/libsnappy.so
> /usr/lib/libsnappy.so.1
> /usr/lib/libsnappy.so.1.1.3
> /usr/local/lib/libsnappy.so
> /usr/local/lib/libsnappy.so.1
> /usr/local/lib/libsnappy.so.1.1.3
>
> I tried to add this on my hbase-env.xml:
> export
> HBASE_LIBRARY_PATH=/home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64
>
> Before I was trying with doing export on the command line directly
> since it seems the hbase script is taking that into consideration too.
>
> I have not yet put the hbase.regionserver.codecs line since I still
> need to use my cluster until I get snappy working. On the hbase/lib
> directory I have snappy-java-1.0.3.2.jar.
>
>
> Should snappy be installed within hbase? Or should it be in hadoop?
> I'm not sure anymore.
>
> But it's still not working. So I tried the small code below:
>
> import java.util.StringTokenizer;
>
> public class Test
> {
>   static {
>     try {
>       System.loadLibrary("snappy");
>       System.loadLibrary("hadoopsnappy");
>       System.out.println ("Snappy native library is available");
>     } catch (UnsatisfiedLinkError ex) {
>         ex.printStackTrace();
>     }
>   }
>
>         public static void main (String [] args)
>         {
>                 System.out.println ("Coucou");
> String property = System.getProperty("java.library.path");
> StringTokenizer parser = new StringTokenizer(property, ";");
> while (parser.hasMoreTokens()) {
>     System.err.println(parser.nextToken());
>     }
>         }
> }
>
>
> This code is from org.apache.hadoop.io.compress.snappy.LoadSnappy.
+
Kevin Odell 2012-12-03, 14:09
+
Kevin Odell 2012-12-03, 14:37
+
Jean-Marc Spaggiari 2012-12-03, 14:56
+
Jean-Marc Spaggiari 2012-12-03, 15:15
+
Kevin Odell 2012-12-03, 15:19
+
Jean-Marc Spaggiari 2012-12-03, 15:50
+
surfer 2012-12-04, 06:29
+
Jean-Marc Spaggiari 2012-12-04, 12:27
+
ac@...) 2012-12-03, 14:22
+
ac@...) 2012-12-03, 14:29
+
Stack 2012-12-03, 18:20
+
Jean-Marc Spaggiari 2012-12-03, 18:55
+
Jean-Marc Spaggiari 2012-12-03, 19:48
+
Stack 2012-12-03, 20:15
+
ac@...) 2012-12-02, 06:16
+
Håvard Wahl Kongsgård 2012-12-02, 23:52
+
Arati Patro 2012-12-03, 05:41
+
Mohamed Ibrahim 2012-12-03, 02:10