Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Setting up Snappy compression in Hadoop


Copy link to this message
-
Re: Setting up Snappy compression in Hadoop
Please take a look at http://hbase.apache.org/book.html#snappy.compression

Cheers
On Wed, Jan 1, 2014 at 8:05 AM, Amit Sela <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> I'm running on Hadoop 1.0.4 and I'd like to use Snappy for map output
> compression.
> I'm adding the configurations:
>
> configuration.setBoolean("mapred.compress.map.output", true);
> configuration.set("mapred.map.output.compression.codec",
> "org.apache.hadoop.io.compress.SnappyCodec");
>
> And I've added libsnappy.so.1 to $HADOOP_HOME/lib/native/Linux-amd64-64/
>
> Still, all map tasks fail with "native snappy library not available".
>
> Could anyone elaborate on how to install Snappy for Hadoop ?
>
> Thanks,
>
> Amit.
>