Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> map/reduce of compressed Avro

Copy link to this message
Re: map/reduce of compressed Avro
You don't need to do anything special to accept compressed Avro files as
input, as it's detected automatically and decompressed transparently. M/R
jobs support all codecs that the Java implementation supports; at the
moment I think that's deflate, snappy and bzip2.

If you want to generate compressed output, use
FileOutputFormat.setCompressOutput(job, true);

On 22 April 2013 02:26, nir_zamir <[EMAIL PROTECTED]> wrote:

> Hi,
> Does anyone know if/how it's possible to get compressed Avro files as an
> input to a M/R job?
> If so, which codecs are supported?
> Thanks,
> Nir
> --
> View this message in context:
> http://apache-avro.679487.n3.nabble.com/map-reduce-of-compressed-Avro-tp4026947.html
> Sent from the Avro - Users mailing list archive at Nabble.com.