Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase, mail # user - How to install Snappy?


+
Jean-Marc Spaggiari 2012-12-01, 23:52
+
Jean-Marc Spaggiari 2012-12-01, 23:57
+
Jean-Marc Spaggiari 2012-12-02, 13:25
+
surfer 2012-12-03, 07:02
+
Jean-Marc Spaggiari 2012-12-03, 13:24
+
Jean-Marc Spaggiari 2012-12-03, 13:47
+
Kevin Odell 2012-12-03, 14:09
+
Kevin Odell 2012-12-03, 14:37
+
Jean-Marc Spaggiari 2012-12-03, 14:56
+
Jean-Marc Spaggiari 2012-12-03, 15:15
+
Kevin Odell 2012-12-03, 15:19
+
Jean-Marc Spaggiari 2012-12-03, 15:50
+
surfer 2012-12-04, 06:29
+
Jean-Marc Spaggiari 2012-12-04, 12:27
+
ac@...) 2012-12-03, 14:22
+
ac@...) 2012-12-03, 14:29
+
Stack 2012-12-03, 18:20
Copy link to this message
-
Re: How to install Snappy?
Jean-Marc Spaggiari 2012-12-03, 18:55
Sure I will.

JM

2012/12/3, Stack <[EMAIL PROTECTED]>:
> Any chance of an update to
> http://hbase.apache.org/book.html#snappy.compression ?  If someone writes
> it up, I'll stitch it in. Thanks,
> St.Ack
>
>
> On Mon, Dec 3, 2012 at 6:29 AM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Something more about my workaround last time:
>>
>> I used the following steps to test my workaround:
>>
>> 1) cd $HBASE_HOME
>> ./bin/hbase org.apache.hadoop.hbase.util.CompressionTest
>> file:///tmp/testfile lzo
>>
>>
>> 2) hbase shell
>> create 't1', {NAME => 'cf1', COMPRESSION => 'LZO'}
>>
>>
>> You could modify above for your test cases.
>>
>> Good luck
>> Thanks
>> AC
>>
>>
>> On 3 Dec 2012, at 10:22 PM, [EMAIL PROTECTED] wrote:
>>
>> > Hi JM,
>> >
>> > I had experienced similar error when I was installing LZO compression
>> > to
>> RegionServers:
>> >
>> > Below is from my record about installing LZO:
>> > Issue:
>> > java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
>> > ...
>> > 12/11/23 19:03:14 ERROR lzo.LzoCodec: Cannot load native-lzo without
>> native-hadoop
>> > Exception in thread "main" java.lang.RuntimeException: native-lzo
>> library not available
>> > Solution:
>> > (compiled lzo from source), then do these EXTRA steps:
>> > 1)  cp
>> {working_folder_of_lzo}/hadoop-lzo-master/build/native/Linux-amd64-64/lib/*
>> /usr/local/lib/
>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
>> $HBASE_HOME/conf/hbase-env.sh
>> > 3) mkdir -p $HBASE_HOME/build
>> > 4) cp -r {working_folder_of_lzo}/hadoop-lzo-master/build/native
>> $HBASE_HOME/build/native
>> >
>> >
>> >
>> > When looking at your email below,  I saw your log also has "The error
>> I'm getting is java.lang.UnsatisfiedLinkError: no hadoopsnappy in
>> java.library.path."
>> > I think you could try these steps:
>> > 1)  cp {working_folder_your_snappy}/build/native/Linux-amd64-64/lib/*
>> /usr/local/lib/
>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
>> $HBASE_HOME/conf/hbase-env.sh
>> > 3) mkdir -p $HBASE_HOME/build
>> > 4) cp -r {working_folder_your_snappy}/build/native
>> $HBASE_HOME/build/native
>> >
>> >
>> > Good luck.
>> > Thanks
>> > AC
>> >
>> >
>> >
>> > On 3 Dec 2012, at 9:47 PM, Jean-Marc Spaggiari wrote:
>> >
>> >> Ok....
>> >>
>> >> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2 helped
>> >> me and my test program is now working. I'm able to load both
>> >> libraries. Fine.
>> >>
>> >> But the CompressionTest is still not working.
>> >>
>> >> What is very strange is that:
>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
>> available
>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not
>> loaded
>> >>
>> >> It's available, but not loaded.
>> >>
>> >> But from the code:
>> >>  static {
>> >>    try {
>> >>      System.loadLibrary("snappy");
>> >>      System.loadLibrary("hadoopsnappy");
>> >>      LOG.warn("Snappy native library is available");
>> >>      AVAILABLE = true;
>> >>    } catch (UnsatisfiedLinkError ex) {
>> >>      //NOP
>> >>    }
>> >>    LOADED = AVAILABLE;
>> >>    if (LOADED) {
>> >>      LOG.info("Snappy native library loaded");
>> >>    } else {
>> >>      LOG.warn("Snappy native library not loaded");
>> >>    }
>> >>  }
>> >> If "Snappy native library is available" is displayed, that mean
>> >> AVAILABLE = true... And if AVAILABLE = true, then LOADED is set to
>> >> true and Snappy native library loaded must be displayed... But it's
>> >> not... How is this possible?
>> >>
>> >> I have not expected Snappy installation to be such a challenge...
>> >>
>> >> I will continue to dig and summarize the steps when I will be done (If
>> >> I'm able to finish...)
>> >>
>> >> JM
>> >>
>> >> 2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
>> >>> Thanks all for your replies.
>> >>>
>> >>> So, to reply to all in one.
>> >>>
>> >>> I'm not using CD3. I'm using Hadoop  1.0.3 and HBase 0.94.2 directly
>> >>> from the JARs.
>> >>>
>> >>> Here are all the places where I have put the lib:
+
Jean-Marc Spaggiari 2012-12-03, 19:48
+
Stack 2012-12-03, 20:15
+
ac@...) 2012-12-02, 06:16
+
Håvard Wahl Kongsgård 2012-12-02, 23:52
+
Arati Patro 2012-12-03, 05:41
+
Mohamed Ibrahim 2012-12-03, 02:10