Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Setting up Snappy compression in Hadoop


Copy link to this message
-
Re: Setting up Snappy compression in Hadoop
Please take a look at http://hbase.apache.org/book.html#snappy.compression

Cheers
On Wed, Jan 1, 2014 at 8:05 AM, Amit Sela <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I'm running on Hadoop 1.0.4 and I'd like to use Snappy for map output
> compression.
> I'm adding the configurations:
>
> configuration.setBoolean("mapred.compress.map.output", true);
> configuration.set("mapred.map.output.compression.codec",
> "org.apache.hadoop.io.compress.SnappyCodec");
>
> And I've added libsnappy.so.1 to $HADOOP_HOME/lib/native/Linux-amd64-64/
>
> Still, all map tasks fail with "native snappy library not available".
>
> Could anyone elaborate on how to install Snappy for Hadoop ?
>
> Thanks,
>
> Amit.
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB