Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How to install Snappy?


Copy link to this message
-
Re: How to install Snappy?
Hi Kevin,

Thanks for the clarification.

No, it's not what I'm seeing.

Here is what I'm getting:

12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library is available
12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library not loaded
Exception in thread "main" java.lang.RuntimeException: native snappy
library not available
at org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:123)
at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
at org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
at org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:264)
at org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:739)
at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:127)
at org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:118)
at org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:101)
at org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:394)
at org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(CompressionTest.java:108)
at org.apache.hadoop.hbase.util.CompressionTest.main(CompressionTest.java:138)

The most disturbing part is this line:
12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library is available

Followed by this one:
Exception in thread "main" java.lang.RuntimeException: native snappy
library not available

Is it available? Or is it not available?

I looked in the code and I have no idea why the 2nd one is raised.

The code I'm looking at is on the hadoop-snappy site, but the one I
have on my server is in the hadoop the hadoop-core-1.0.3.jar file. So
maybe that the issue and they are different?

I built the hadoop-snappy-0.0.1-SNAPSHOT.jar file too. I placed it on
the lib folder and made sure it was taken first, but still not
working.

So far I think I will stay with GZip until Snappy is integrated on the
HBase files...

JM

2012/12/3, Kevin O'dell <[EMAIL PROTECTED]>:
> Hey JM,
>
>   Sorry for the quick message earlier.  I tracked down the JIRA I was
> referring to: https://issues.apache.org/jira/browse/HBASE-7080
>
> Does this look like what you are seeing in Compression test?
>
> On Mon, Dec 3, 2012 at 9:09 AM, Kevin O'dell
> <[EMAIL PROTECTED]>wrote:
>
>> There is a compression test JIRA right now.  What are you seeing?
>>
>>
>> On Mon, Dec 3, 2012 at 8:47 AM, Jean-Marc Spaggiari <
>> [EMAIL PROTECTED]> wrote:
>>
>>> Ok....
>>>
>>> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2 helped
>>> me and my test program is now working. I'm able to load both
>>> libraries. Fine.
>>>
>>> But the CompressionTest is still not working.
>>>
>>> What is very strange is that:
>>> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
>>> available
>>> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not
>>> loaded
>>>
>>> It's available, but not loaded.
>>>
>>> But from the code:
>>>   static {
>>>     try {
>>>       System.loadLibrary("snappy");
>>>       System.loadLibrary("hadoopsnappy");
>>>       LOG.warn("Snappy native library is available");
>>>       AVAILABLE = true;
>>>     } catch (UnsatisfiedLinkError ex) {
>>>       //NOP
>>>     }
>>>     LOADED = AVAILABLE;
>>>     if (LOADED) {
>>>       LOG.info("Snappy native library loaded");
>>>     } else {
>>>       LOG.warn("Snappy native library not loaded");
>>>     }
>>>   }
>>> If "Snappy native library is available" is displayed, that mean
>>> AVAILABLE = true... And if AVAILABLE = true, then LOADED is set to
>>> true and Snappy native library loaded must be displayed... But it's
>>> not... How is this possible?
>>>
>>> I have not expected Snappy installation to be such a challenge...
>>>
>>> I will continue to dig and summarize the steps when I will be done (If
>>> I'm able to finish...)
>>>
>>> JM
>>>
>>> 2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>: