Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> How to install Snappy?


+
Jean-Marc Spaggiari 2012-12-01, 23:52
+
Jean-Marc Spaggiari 2012-12-01, 23:57
+
Jean-Marc Spaggiari 2012-12-02, 13:25
+
surfer 2012-12-03, 07:02
+
Jean-Marc Spaggiari 2012-12-03, 13:24
+
Jean-Marc Spaggiari 2012-12-03, 13:47
+
Kevin Odell 2012-12-03, 14:09
+
Kevin Odell 2012-12-03, 14:37
+
Jean-Marc Spaggiari 2012-12-03, 14:56
+
Jean-Marc Spaggiari 2012-12-03, 15:15
+
Kevin Odell 2012-12-03, 15:19
+
Jean-Marc Spaggiari 2012-12-03, 15:50
+
surfer 2012-12-04, 06:29
+
Jean-Marc Spaggiari 2012-12-04, 12:27
+
ac@...) 2012-12-03, 14:22
+
ac@...) 2012-12-03, 14:29
+
Stack 2012-12-03, 18:20
+
Jean-Marc Spaggiari 2012-12-03, 18:55
+
Jean-Marc Spaggiari 2012-12-03, 19:48
Copy link to this message
-
Re: How to install Snappy?
Thank you JM.  Let me fold it in....
St.Ack
On Mon, Dec 3, 2012 at 11:48 AM, Jean-Marc Spaggiari <
[EMAIL PROTECTED]> wrote:

> Done. HBASE-7264 created. I have added few more details about the
> steps to follow to install Snappy on HBase 0.94.x.
>
> I don't know how to build the documentation locally, so I'm not 100%
> sure about thhe XML structure. I did cut&past so it should be good...
>
> JM
>
> 2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
> > Sure I will.
> >
> > JM
> >
> > 2012/12/3, Stack <[EMAIL PROTECTED]>:
> >> Any chance of an update to
> >> http://hbase.apache.org/book.html#snappy.compression ?  If someone
> writes
> >> it up, I'll stitch it in. Thanks,
> >> St.Ack
> >>
> >>
> >> On Mon, Dec 3, 2012 at 6:29 AM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
> >>
> >>> Hi,
> >>>
> >>> Something more about my workaround last time:
> >>>
> >>> I used the following steps to test my workaround:
> >>>
> >>> 1) cd $HBASE_HOME
> >>> ./bin/hbase org.apache.hadoop.hbase.util.CompressionTest
> >>> file:///tmp/testfile lzo
> >>>
> >>>
> >>> 2) hbase shell
> >>> create 't1', {NAME => 'cf1', COMPRESSION => 'LZO'}
> >>>
> >>>
> >>> You could modify above for your test cases.
> >>>
> >>> Good luck
> >>> Thanks
> >>> AC
> >>>
> >>>
> >>> On 3 Dec 2012, at 10:22 PM, [EMAIL PROTECTED] wrote:
> >>>
> >>> > Hi JM,
> >>> >
> >>> > I had experienced similar error when I was installing LZO compression
> >>> > to
> >>> RegionServers:
> >>> >
> >>> > Below is from my record about installing LZO:
> >>> > Issue:
> >>> > java.lang.UnsatisfiedLinkError: no gplcompression in
> java.library.path
> >>> > ...
> >>> > 12/11/23 19:03:14 ERROR lzo.LzoCodec: Cannot load native-lzo without
> >>> native-hadoop
> >>> > Exception in thread "main" java.lang.RuntimeException: native-lzo
> >>> library not available
> >>> > Solution:
> >>> > (compiled lzo from source), then do these EXTRA steps:
> >>> > 1)  cp
> >>>
> {working_folder_of_lzo}/hadoop-lzo-master/build/native/Linux-amd64-64/lib/*
> >>> /usr/local/lib/
> >>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
> >>> $HBASE_HOME/conf/hbase-env.sh
> >>> > 3) mkdir -p $HBASE_HOME/build
> >>> > 4) cp -r {working_folder_of_lzo}/hadoop-lzo-master/build/native
> >>> $HBASE_HOME/build/native
> >>> >
> >>> >
> >>> >
> >>> > When looking at your email below,  I saw your log also has "The error
> >>> I'm getting is java.lang.UnsatisfiedLinkError: no hadoopsnappy in
> >>> java.library.path."
> >>> > I think you could try these steps:
> >>> > 1)  cp {working_folder_your_snappy}/build/native/Linux-amd64-64/lib/*
> >>> /usr/local/lib/
> >>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
> >>> $HBASE_HOME/conf/hbase-env.sh
> >>> > 3) mkdir -p $HBASE_HOME/build
> >>> > 4) cp -r {working_folder_your_snappy}/build/native
> >>> $HBASE_HOME/build/native
> >>> >
> >>> >
> >>> > Good luck.
> >>> > Thanks
> >>> > AC
> >>> >
> >>> >
> >>> >
> >>> > On 3 Dec 2012, at 9:47 PM, Jean-Marc Spaggiari wrote:
> >>> >
> >>> >> Ok....
> >>> >>
> >>> >> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2
> >>> >> helped
> >>> >> me and my test program is now working. I'm able to load both
> >>> >> libraries. Fine.
> >>> >>
> >>> >> But the CompressionTest is still not working.
> >>> >>
> >>> >> What is very strange is that:
> >>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
> >>> available
> >>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not
> >>> loaded
> >>> >>
> >>> >> It's available, but not loaded.
> >>> >>
> >>> >> But from the code:
> >>> >>  static {
> >>> >>    try {
> >>> >>      System.loadLibrary("snappy");
> >>> >>      System.loadLibrary("hadoopsnappy");
> >>> >>      LOG.warn("Snappy native library is available");
> >>> >>      AVAILABLE = true;
> >>> >>    } catch (UnsatisfiedLinkError ex) {
> >>> >>      //NOP
> >>> >>    }
> >>> >>    LOADED = AVAILABLE;
> >>> >>    if (LOADED) {
> >>> >>      LOG.info("Snappy native library loaded");
> >>> >>    } else {
+
ac@...) 2012-12-02, 06:16
+
Håvard Wahl Kongsgård 2012-12-02, 23:52
+
Arati Patro 2012-12-03, 05:41
+
Mohamed Ibrahim 2012-12-03, 02:10