Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Should compression with Avro import work?


Copy link to this message
-
Should compression with Avro import work?
Hi,

I've tried using sqoop import with Avro and it works beautifully. But
I've tried it with compression which according to the documentation
should work. It didn't for me so before I go any further: Is this a
problem in the documentation?

I briefly looked at the AvroOutputFormat and it doesn't do any
compression as far as I can tell. The job.xml is certainly correct in
that it mentions mapred.output.compress=true.

Removing the --as-avrodatafile option gives me a compressed text file
so I'm pretty sure I've got the command line right.

Cheers,
Lars