Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> How to install Snappy?


Copy link to this message
-
Re: How to install Snappy?
Ok, I got it!!!!

I had to copy the hadoop native libs into the hbase native libs directory!

Now I get a SUCCESS when I'm doint the CompressionTest...

I'm not 100% sure that it's the only think which was missing because I
have done so many modifications in the last 3 days...

So I will start from a blank 0.94.3 jar and re-do all the steps to
make sure it's just the native libs which need to be copied.

I was close to surrender ;)

JM

2012/12/3, Jean-Marc Spaggiari <[EMAIL PROTECTED]>:
> Hi Kevin,
>
> Thanks for the clarification.
>
> No, it's not what I'm seeing.
>
> Here is what I'm getting:
>
> 12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library is
> available
> 12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library not loaded
> Exception in thread "main" java.lang.RuntimeException: native snappy
> library not available
> at
> org.apache.hadoop.io.compress.SnappyCodec.getCompressorType(SnappyCodec.java:123)
> at
> org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:100)
> at
> org.apache.hadoop.io.compress.CodecPool.getCompressor(CodecPool.java:112)
> at
> org.apache.hadoop.hbase.io.hfile.Compression$Algorithm.getCompressor(Compression.java:264)
> at
> org.apache.hadoop.hbase.io.hfile.HFileBlock$Writer.<init>(HFileBlock.java:739)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.finishInit(HFileWriterV2.java:127)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2.<init>(HFileWriterV2.java:118)
> at
> org.apache.hadoop.hbase.io.hfile.HFileWriterV2$WriterFactoryV2.createWriter(HFileWriterV2.java:101)
> at
> org.apache.hadoop.hbase.io.hfile.HFile$WriterFactory.create(HFile.java:394)
> at
> org.apache.hadoop.hbase.util.CompressionTest.doSmokeTest(CompressionTest.java:108)
> at
> org.apache.hadoop.hbase.util.CompressionTest.main(CompressionTest.java:138)
>
> The most disturbing part is this line:
> 12/12/03 09:40:42 WARN snappy.LoadSnappy: Snappy native library is
> available
>
> Followed by this one:
> Exception in thread "main" java.lang.RuntimeException: native snappy
> library not available
>
> Is it available? Or is it not available?
>
> I looked in the code and I have no idea why the 2nd one is raised.
>
> The code I'm looking at is on the hadoop-snappy site, but the one I
> have on my server is in the hadoop the hadoop-core-1.0.3.jar file. So
> maybe that the issue and they are different?
>
> I built the hadoop-snappy-0.0.1-SNAPSHOT.jar file too. I placed it on
> the lib folder and made sure it was taken first, but still not
> working.
>
> So far I think I will stay with GZip until Snappy is integrated on the
> HBase files...
>
> JM
>
> 2012/12/3, Kevin O'dell <[EMAIL PROTECTED]>:
>> Hey JM,
>>
>>   Sorry for the quick message earlier.  I tracked down the JIRA I was
>> referring to: https://issues.apache.org/jira/browse/HBASE-7080
>>
>> Does this look like what you are seeing in Compression test?
>>
>> On Mon, Dec 3, 2012 at 9:09 AM, Kevin O'dell
>> <[EMAIL PROTECTED]>wrote:
>>
>>> There is a compression test JIRA right now.  What are you seeing?
>>>
>>>
>>> On Mon, Dec 3, 2012 at 8:47 AM, Jean-Marc Spaggiari <
>>> [EMAIL PROTECTED]> wrote:
>>>
>>>> Ok....
>>>>
>>>> This: http://code.google.com/p/hadoop-snappy/issues/detail?id=2 helped
>>>> me and my test program is now working. I'm able to load both
>>>> libraries. Fine.
>>>>
>>>> But the CompressionTest is still not working.
>>>>
>>>> What is very strange is that:
>>>> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library is
>>>> available
>>>> 12/12/03 08:44:24 WARN snappy.LoadSnappy: Snappy native library not
>>>> loaded
>>>>
>>>> It's available, but not loaded.
>>>>
>>>> But from the code:
>>>>   static {
>>>>     try {
>>>>       System.loadLibrary("snappy");
>>>>       System.loadLibrary("hadoopsnappy");
>>>>       LOG.warn("Snappy native library is available");
>>>>       AVAILABLE = true;
>>>>     } catch (UnsatisfiedLinkError ex) {
>>>>       //NOP
>>>>     }
>>>>     LOADED = AVAILABLE;