Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase >> mail # user >> Enable Snappy compression - not able to load the libs on startup


Copy link to this message
-
Re: Enable Snappy compression - not able to load the libs on startup
Perhaps the pre-compiled set does not work against the version of libs
in your ArchLinux. We've noticed this to be the case between CentOS 5
and 6 versions too (5 doesn't pick up the Snappy codec for some
reason).

Try recompiling them on the hadoop side (ant compile-native, etc.).
For a loose dependency set to compile the natives, see
http://wiki.apache.org/hadoop/QwertyManiac/BuildingHadoopTrunk.
Alternatively, you can also run the CDH build script under
$HADOOP_HOME/cloudera/do-release-build to get it going automatically
and producing a new tarball.

On Mon, Jul 9, 2012 at 1:51 AM, Arvid Warnecke <[EMAIL PROTECTED]> wrote:
> Hello Paul,
>
> On Sun, Jul 08, 2012 at 06:45:46PM +0200, Paul Cavallaro wrote:
>> On Sun, Jul 8, 2012 at 9:48 AM, Arvid Warnecke <[EMAIL PROTECTED]> wrote:
>> > I already found some old entries from mailinglists and articles at
>> > Cloudera how to use the Snappy library from Hadoop in HBase, but it does
>> > not seem to work for me.
>> >
>> > I installed Hadoop and HBase from the tarballs, because there are no
>> > packages available for Arch Linux. Everything worked fine, but I am not
>> > able to use any compression for my tables.
>> >
>> > When I use
>> >
>> > hbase> create 'table', {NAME=>'fam', COMPRESSION=>'snappy'}
>> >
>> I would first ask if you've installed the native snappy libraries on the
>> machine?
>>
>> http://hbase.apache.org/book/snappy.compression.html
>>
>> That seems to be the likely culprit here.
>>
> No, I did not. I installed Hadoop via Cloudera tarball. There are libs
> for different compressions available at $HADOOP_HOME/lib/native.
> Is there a difference?
>
> Cheers,
> Arvid
>
> --
> [ Arvid Warnecke ][ arvid (at) nostalgix (dot) org ]
> [ IRC/OPN: "madhatter" ][ http://www.nostalgix.org ]
> ---[  ThreePiO was right: Let the Wookiee win.  ]---

--
Harsh J
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB