Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> map/reduce of compressed Avro


+
nir_zamir 2013-04-22, 09:26
+
Martin Kleppmann 2013-04-22, 14:52
Copy link to this message
-
Re: map/reduce of compressed Avro
Thanks Martin.

What will happen if I try to use an indexed LZO-compressed avro file? Will
it work and utilize the index to allow multiple mappers?

I think that for Snappy for example, the file is splittable and can use
multiple mappers, but I haven't tested it yet - would be glad if anyone has
any experience with that.

Thanks!
Nir.

--
View this message in context: http://apache-avro.679487.n3.nabble.com/map-reduce-of-compressed-Avro-tp4026947p4027009.html
Sent from the Avro - Users mailing list archive at Nabble.com.
+
Martin Kleppmann 2013-04-23, 10:38
+
Scott Carey 2013-04-29, 22:47
+
Enns, Steven 2013-04-29, 22:53
+
nir_zamir 2013-05-01, 07:11
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB