Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> has bzip2 compression been deprecated?


Copy link to this message
-
has bzip2 compression been deprecated?
Hi,

I'm trying to work out which compression algorithm I should be using in my MapReduce jobs.  It seems to me that the best solution is a compromise between speed, efficiency and splittability. The only compression algorithm to handle file splits (according to Hadoop: The Definitive Guide 2nd edition p78 etc) is bzip2, at the expense of compression speed.

However, I see from the documentation at http://hadoop.apache.org/common/docs/current/native_libraries.html that the bzip2 library is no longer mentioned, and hasn't been since version 0.20.0, see http://hadoop.apache.org/common/docs/r0.20.0/native_libraries.html - however the bzip2 Codec is still in the API at http://hadoop.apache.org/common/docs/current/api/org/apache/hadoop/io/compress/BZip2Codec.html.

Has bzip2 support been removed from Hadoop, or will it be removed soon?

Thanks,

Tony

**********************************************************************

This email and any attachments are confidential, protected by copyright and may be legally privileged.  If you are not the intended recipient, then the dissemination or copying of this email is prohibited. If you have received this in error, please notify the sender by replying by email and then delete the email completely from your system.  Neither Sporting Index nor the sender accepts responsibility for any virus, or any other defect which might affect any computer or IT system into which the email is received and/or opened.  It is the responsibility of the recipient to scan the email and no responsibility is accepted for any loss or damage arising in any way from receipt or use of this email.  Sporting Index Ltd is a company registered in England and Wales with company number 2636842, whose registered office is at Brookfield House, Green Lane, Ivinghoe, Leighton Buzzard, LU7 9ES.  Sporting Index Ltd is authorised and regulated by the UK Financial Services Authority (reg. no. 150404). Any financial promotion contained herein has been issued
and approved by Sporting Index Ltd.

Outbound email has been scanned for viruses and SPAM
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB