Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Sqoop >> mail # user >> Should compression with Avro import work?


Copy link to this message
-
Should compression with Avro import work?
Hi,

I've tried using sqoop import with Avro and it works beautifully. But
I've tried it with compression which according to the documentation
should work. It didn't for me so before I go any further: Is this a
problem in the documentation?

I briefly looked at the AvroOutputFormat and it doesn't do any
compression as far as I can tell. The job.xml is certainly correct in
that it mentions mapred.output.compress=true.

Removing the --as-avrodatafile option gives me a compressed text file
so I'm pretty sure I've got the command line right.

Cheers,
Lars
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB