Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop, mail # user - Should compression with Avro import work?

Copy link to this message
Should compression with Avro import work?
Lars Francke 2012-01-18, 15:16

I've tried using sqoop import with Avro and it works beautifully. But
I've tried it with compression which according to the documentation
should work. It didn't for me so before I go any further: Is this a
problem in the documentation?

I briefly looked at the AvroOutputFormat and it doesn't do any
compression as far as I can tell. The job.xml is certainly correct in
that it mentions mapred.output.compress=true.

Removing the --as-avrodatafile option gives me a compressed text file
so I'm pretty sure I've got the command line right.