Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - How to install Snappy?


+
Jean-Marc Spaggiari 2012-12-01, 23:52
+
Jean-Marc Spaggiari 2012-12-01, 23:57
+
Jean-Marc Spaggiari 2012-12-02, 13:25
+
surfer 2012-12-03, 07:02
+
Jean-Marc Spaggiari 2012-12-03, 13:24
+
Jean-Marc Spaggiari 2012-12-03, 13:47
+
Kevin Odell 2012-12-03, 14:09
Copy link to this message
-
Re: How to install Snappy?
Kevin O'dell 2012-12-03, 14:37
Hey JM,

  Sorry for the quick message earlier.  I tracked down the JIRA I was
referring to: https://issues.apache.org/jira/browse/HBASE-7080

Does this look like what you are seeing in Compression test?

On Mon, Dec 3, 2012 at 9:09 AM, Kevin O'dell <[EMAIL PROTECTED]>wrote:

> There is a compression test JIRA right now.  What are you seeing?
>
>
> On Mon, Dec 3, 2012 at 8:47 AM, Jean-Marc Spaggiari <
> [EMAIL PROTECTED]> wrote:
>
>> Ok....
>>
>> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2 helped
>> me and my test program is now working. I'm able to load both
>> libraries. Fine.
>>
>> But the CompressionTest is still not working.
>>
>> What is very strange is that:
>> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
>> available
>> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not loaded
>>
>> It's available, but not loaded.
>>
>> But from the code:
>>   static {
>>     try {
>>       System.loadLibrary("snappy");
>>       System.loadLibrary("hadoopsnappy");
>>       LOG.warn("Snappy native library is available");
>>       AVAILABLE = true;
>>     } catch (UnsatisfiedLinkError ex) {
>>       //NOP
>>     }
>>     LOADED = AVAILABLE;
>>     if (LOADED) {
>>       LOG.info("Snappy native library loaded");
>>     } else {
>>       LOG.warn("Snappy native library not loaded");
>>     }
>>   }
>> If "Snappy native library is available" is displayed, that mean
>> AVAILABLE = true... And if AVAILABLE = true, then LOADED is set to
>> true and Snappy native library loaded must be displayed... But it's
>> not... How is this possible?
>>
>> I have not expected Snappy installation to be such a challenge...
>>
>> I will continue to dig and summarize the steps when I will be done (If
>> I'm able to finish...)
>>
>> JM
>>
>> 2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
>> > Thanks all for your replies.
>> >
>> > So, to reply to all in one.
>> >
>> > I'm not using CD3. I'm using Hadoop  1.0.3 and HBase 0.94.2 directly
>> > from the JARs.
>> >
>> > Here are all the places where I have put the lib:
>> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so
>> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so.1
>> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-amd64-64/libsnappy.so.1.1.3
>> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so
>> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so.1
>> > /home/hadoop/hadoop-1.0.3/lib/native/Linux-i386-32/libsnappy.so.1.1.3
>> > /home/hbase/hbase-0.94.2/lib/native/libsnappy.so
>> > /home/hbase/hbase-0.94.2/lib/native/libsnappy.so.1
>> > /home/hbase/hbase-0.94.2/lib/native/libsnappy.so.1.1.3
>> > /home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so
>> > /home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so.1
>> > /home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64/libsnappy.so.1.1.3
>> > /home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so
>> > /home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so.1
>> > /home/hbase/hbase-0.94.2/lib/native/Linux-i386-32/libsnappy.so.1.1.3
>> > /lib/x86_64-linux-gnu/libsnappy.so
>> > /usr/lib/libsnappy.so
>> > /usr/lib/libsnappy.so.1
>> > /usr/lib/libsnappy.so.1.1.3
>> > /usr/local/lib/libsnappy.so
>> > /usr/local/lib/libsnappy.so.1
>> > /usr/local/lib/libsnappy.so.1.1.3
>> >
>> > I tried to add this on my hbase-env.xml:
>> > export
>> > HBASE_LIBRARY_PATH=/home/hbase/hbase-0.94.2/lib/native/Linux-amd64-64
>> >
>> > Before I was trying with doing export on the command line directly
>> > since it seems the hbase script is taking that into consideration too.
>> >
>> > I have not yet put the hbase.regionserver.codecs line since I still
>> > need to use my cluster until I get snappy working. On the hbase/lib
>> > directory I have snappy-java-1.0.3.2.jar.
>> >
>> >
>> > Should snappy be installed within hbase? Or should it be in hadoop?
>> > I'm not sure anymore.
>> >
>> > But it's still not working. So I tried the small code below:

Kevin O'Dell
Customer Operations Engineer, Cloudera
+
Jean-Marc Spaggiari 2012-12-03, 14:56
+
Jean-Marc Spaggiari 2012-12-03, 15:15
+
Kevin Odell 2012-12-03, 15:19
+
Jean-Marc Spaggiari 2012-12-03, 15:50
+
surfer 2012-12-04, 06:29
+
Jean-Marc Spaggiari 2012-12-04, 12:27
+
ac@...) 2012-12-03, 14:22
+
ac@...) 2012-12-03, 14:29
+
Stack 2012-12-03, 18:20
+
Jean-Marc Spaggiari 2012-12-03, 18:55
+
Jean-Marc Spaggiari 2012-12-03, 19:48
+
Stack 2012-12-03, 20:15
+
ac@...) 2012-12-02, 06:16
+
Håvard Wahl Kongsgård 2012-12-02, 23:52
+
Arati Patro 2012-12-03, 05:41
+
Mohamed Ibrahim 2012-12-03, 02:10