Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Hadoop stream gzipped file with AvroAsTextInputFormat


Copy link to this message
-
Hadoop stream gzipped file with AvroAsTextInputFormat


I have successfully stream Avro data file to Python mrjobs using the
library AvroAsTextInputFormat

-inputformat
org.apache.avro.mapred.AvroAsTextInputFormat

However, unlike text file,
it does not seems to handle gzipped file automatically. What can I do to
stream a gzipped Avro file?

Wai Yip
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB