Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HBase, mail # user - Compression


Copy link to this message
-
Re: Compression
Otis Gospodnetic 2011-09-14, 15:10
We use lzo, as do some of our customers.  We have not tried Snappy yet, but all feedback I ever saw about it was positive.

Otis
----
Sematext :: http://sematext.com/ :: Solr - Lucene - Hadoop - HBase
Hadoop ecosystem search :: http://search-hadoop.com/

>________________________________
>From: Wayne <[EMAIL PROTECTED]>
>To: [EMAIL PROTECTED]
>Sent: Wednesday, September 14, 2011 8:33 AM
>Subject: Compression
>
>I wanted to do a poll on what compression libraries people are using and
>why. We currently use lzo but are considering other alternatives for various
>reasons. We would like to move to CDH3 but adding lzo ourselves is a hassle
>we are not looking to take on. It kind of defeats the purpose os using CDH3
>to begin with. We current run 20.0 append.
>
>I know there are a lot of variables that affect the best decision, but we
>are looking for general trends in the community.
>
>Is lzo still the most recommended? Is there benefit in using the lzo
>professional library and does anyone use this?
>Is snappy just as good as lzo and a lot easier to deal with in term of node
>build/releases?
>Does zlib/gzip have any traction?
>
>Compression ratios are important but as always performance/speed is our
>biggest requirement. What are people using and why? Where is the momentum
>going? Compression is a huge benefit of hadoop/hbase and having high
>compression ratios with solid performance is a major benefit.
>
>Any recommendations would be appreciated.
>
>Thanks.
>
>
>