Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> map/reduce of compressed Avro


+
nir_zamir 2013-04-22, 09:26
+
Martin Kleppmann 2013-04-22, 14:52
+
nir_zamir 2013-04-23, 06:47
+
Martin Kleppmann 2013-04-23, 10:38
+
Scott Carey 2013-04-29, 22:47
+
Enns, Steven 2013-04-29, 22:53
Copy link to this message
-
Re: map/reduce of compressed Avro
Thanks.

If the compression codec doesn't matter, what does it mean that Avro added
support for Snappy codec?
If I need the files to be used as input for a M/R, I guess the avro module
should be able to decompress each block and extract the objects. Does it
make sense?

So are you saying that in this case I can use a non-splittable codec (like
deflate)?

Thanks
Nir

--
View this message in context: http://apache-avro.679487.n3.nabble.com/map-reduce-of-compressed-Avro-tp4026947p4027170.html
Sent from the Avro - Users mailing list archive at Nabble.com.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB