Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
HBase >> mail # user >> How to install Snappy?


+
Jean-Marc Spaggiari 2012-12-01, 23:52
+
Jean-Marc Spaggiari 2012-12-01, 23:57
+
Jean-Marc Spaggiari 2012-12-02, 13:25
+
surfer 2012-12-03, 07:02
+
Jean-Marc Spaggiari 2012-12-03, 13:24
+
Jean-Marc Spaggiari 2012-12-03, 13:47
+
Kevin Odell 2012-12-03, 14:09
+
Kevin Odell 2012-12-03, 14:37
+
Jean-Marc Spaggiari 2012-12-03, 14:56
+
Jean-Marc Spaggiari 2012-12-03, 15:15
+
Kevin Odell 2012-12-03, 15:19
+
Jean-Marc Spaggiari 2012-12-03, 15:50
+
surfer 2012-12-04, 06:29
+
Jean-Marc Spaggiari 2012-12-04, 12:27
+
ac@...) 2012-12-03, 14:22
+
ac@...) 2012-12-03, 14:29
+
Stack 2012-12-03, 18:20
Copy link to this message
-
Re: How to install Snappy?
Sure I will.

JM

2012/12/3, Stack <[EMAIL PROTECTED]>:
> Any chance of an update to
> http://hbase.apache.org/book.html#snappy.compression ?  If someone writes
> it up, I'll stitch it in. Thanks,
> St.Ack
>
>
> On Mon, Dec 3, 2012 at 6:29 AM, [EMAIL PROTECTED] <[EMAIL PROTECTED]> wrote:
>
>> Hi,
>>
>> Something more about my workaround last time:
>>
>> I used the following steps to test my workaround:
>>
>> 1) cd $HBASE_HOME
>> ./bin/hbase org.apache.hadoop.hbase.util.CompressionTest
>> file:///tmp/testfile lzo
>>
>>
>> 2) hbase shell
>> create 't1', {NAME => 'cf1', COMPRESSION => 'LZO'}
>>
>>
>> You could modify above for your test cases.
>>
>> Good luck
>> Thanks
>> AC
>>
>>
>> On 3 Dec 2012, at 10:22 PM, [EMAIL PROTECTED] wrote:
>>
>> > Hi JM,
>> >
>> > I had experienced similar error when I was installing LZO compression
>> > to
>> RegionServers:
>> >
>> > Below is from my record about installing LZO:
>> > Issue:
>> > java.lang.UnsatisfiedLinkError: no gplcompression in java.library.path
>> > ...
>> > 12/11/23 19:03:14 ERROR lzo.LzoCodec: Cannot load native-lzo without
>> native-hadoop
>> > Exception in thread "main" java.lang.RuntimeException: native-lzo
>> library not available
>> > Solution:
>> > (compiled lzo from source), then do these EXTRA steps:
>> > 1)  cp
>> {working_folder_of_lzo}/hadoop-lzo-master/build/native/Linux-amd64-64/lib/*
>> /usr/local/lib/
>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
>> $HBASE_HOME/conf/hbase-env.sh
>> > 3) mkdir -p $HBASE_HOME/build
>> > 4) cp -r {working_folder_of_lzo}/hadoop-lzo-master/build/native
>> $HBASE_HOME/build/native
>> >
>> >
>> >
>> > When looking at your email below,  I saw your log also has "The error
>> I'm getting is java.lang.UnsatisfiedLinkError: no hadoopsnappy in
>> java.library.path."
>> > I think you could try these steps:
>> > 1)  cp {working_folder_your_snappy}/build/native/Linux-amd64-64/lib/*
>> /usr/local/lib/
>> > 2) echo export HBASE_LIBRARY_PATH=/usr/local/lib/ >>
>> $HBASE_HOME/conf/hbase-env.sh
>> > 3) mkdir -p $HBASE_HOME/build
>> > 4) cp -r {working_folder_your_snappy}/build/native
>> $HBASE_HOME/build/native
>> >
>> >
>> > Good luck.
>> > Thanks
>> > AC
>> >
>> >
>> >
>> > On 3 Dec 2012, at 9:47 PM, Jean-Marc Spaggiari wrote:
>> >
>> >> Ok....
>> >>
>> >> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2 helped
>> >> me and my test program is now working. I'm able to load both
>> >> libraries. Fine.
>> >>
>> >> But the CompressionTest is still not working.
>> >>
>> >> What is very strange is that:
>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
>> available
>> >> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not
>> loaded
>> >>
>> >> It's available, but not loaded.
>> >>
>> >> But from the code:
>> >>  static {
>> >>    try {
>> >>      System.loadLibrary("snappy");
>> >>      System.loadLibrary("hadoopsnappy");
>> >>      LOG.warn("Snappy native library is available");
>> >>      AVAILABLE = true;
>> >>    } catch (UnsatisfiedLinkError ex) {
>> >>      //NOP
>> >>    }
>> >>    LOADED = AVAILABLE;
>> >>    if (LOADED) {
>> >>      LOG.info("Snappy native library loaded");
>> >>    } else {
>> >>      LOG.warn("Snappy native library not loaded");
>> >>    }
>> >>  }
>> >> If "Snappy native library is available" is displayed, that mean
>> >> AVAILABLE = true... And if AVAILABLE = true, then LOADED is set to
>> >> true and Snappy native library loaded must be displayed... But it's
>> >> not... How is this possible?
>> >>
>> >> I have not expected Snappy installation to be such a challenge...
>> >>
>> >> I will continue to dig and summarize the steps when I will be done (If
>> >> I'm able to finish...)
>> >>
>> >> JM
>> >>
>> >> 2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
>> >>> Thanks all for your replies.
>> >>>
>> >>> So, to reply to all in one.
>> >>>
>> >>> I'm not using CD3. I'm using Hadoop  1.0.3 and HBase 0.94.2 directly
>> >>> from the JARs.
>> >>>
>> >>> Here are all the places where I have put the lib:
+
Jean-Marc Spaggiari 2012-12-03, 19:48
+
Stack 2012-12-03, 20:15
+
ac@...) 2012-12-02, 06:16
+
Håvard Wahl Kongsgård 2012-12-02, 23:52
+
Arati Patro 2012-12-03, 05:41
+
Mohamed Ibrahim 2012-12-03, 02:10
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB