Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How to install Snappy?


Copy link to this message
-
Re: How to install Snappy?
Thanks all for your replies.

So, to reply to all in one.

I'm not using CD3. I'm using Hadoop  1.0.3 and HBase 0.94.2 directly
from the JARs.

Here are all the places where I have put the lib:
/home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so
/home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so.1
/home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so.1.1.3
/home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so
/home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so.1
/home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so.1.1.3
/home/hbase/hbase-0.94.2/lib/native/libsnappy.so
/home/hbase/hbase-0.94.2/lib/native/libsnappy.so.1
/home/hbase/hbase-0.94.2/lib/native/libsnappy.so.1.1.3
/home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so
/home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so.1
/home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so.1.1.3
/home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so
/home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so.1
/home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so.1.1.3
/lib/x86_64-linux-gnu/libsnappy.so
/usr/lib/libsnappy.so
/usr/lib/libsnappy.so.1
/usr/lib/libsnappy.so.1.1.3
/usr/local/lib/libsnappy.so
/usr/local/lib/libsnappy.so.1
/usr/local/lib/libsnappy.so.1.1.3

I tried to add this on my hbase-env.xml:
export HBASE_LIBRARY_PATH=/home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64

Before I was trying with doing export on the command line directly
since it seems the hbase script is taking that into consideration too.

I have not yet put the hbase.regionserver.codecs line since I still
need to use my cluster until I get snappy working. On the hbase/lib
directory I have snappy-java-1.0.3.2.jar.
Should snappy be installed within hbase? Or should it be in hadoop?
I'm not sure anymore.

But it's still not working. So I tried the small code below:

import java.util.StringTokenizer;

public class Test
{
  static {
    try {
      System.loadLibrary("snappy");
      System.loadLibrary("hadoopsnappy");
      System.out.println ("Snappy native library is available");
    } catch (UnsatisfiedLinkError ex) {
        ex.printStackTrace();
    }
  }

        public static void main (String [] args)
        {
                System.out.println ("Coucou");
String property = System.getProperty("java.library.path");
StringTokenizer parser = new StringTokenizer(property, ";");
while (parser.hasMoreTokens()) {
    System.err.println(parser.nextToken());
    }
        }
}
This code is from org.apache.hadoop.io.compress.snappy.LoadSnappy.
The error I'm getting is java.lang.UnsatisfiedLinkError: no
hadoopsnappy in java.library.path.

So the issue is not the snappy lib. It' there and working fine. The
issue is the hadoopsnappy lib which I don't have...

I found it there: http://code.google.com/p/hadoop-snappy/

So I have extracted it with svn checkout
http://hadoop-snappy.googlecode.com/svn/trunk/
hadoop-snappy-read-only, tried to built it with mvn package but it's
failing with something saying "cannot find -ljvm"

So seems my challenge will be to build hadoop-snappy and not to
install snappy which is already there and working...

JM

2012/12/3, surfer <[EMAIL PROTECTED]>:
> hope it helps. this is what I do on apache hadoop 1.0.x and hbase 0.92.y:
> in hbase-site.xml add:
>
> <property>
> <name>hbase.regionserver.codecs</name>
> <value>snappy</value>
> </property>
>
> copy that file into the hadoop conf directory.
>
> in hbase-env.sh:
> export HBASE_LIBRARY_PATH=/pathtoyourhadoop/lib/native/Linux-amd64-64
>
> ( In hbase-env.sh I set also HBASE_HOME, HBASE_CONF_DIR, HADOOP_HOME,
> HADOOP_CONF_DIR but I don't know if they contribute to make snappy
> working...)
>
> in /pathtoyourhadoop/lib/native/Linux-amd64-64 I have:
> libsnappy.a
> libsnappy.so
> libsnappy.so.1
> libsnappy.so.1.1.2
>
> good luck
> giovanni
>
>
>
>
>
>
> On 12/02/2012 02:25 PM, Jean-Marc Spaggiari wrote:
>> So. I spent few hours on that yesterday with no luck.