Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Union in AvroMapper.map: Not in Union??


+
nir_zamir 2013-03-24, 10:23
+
nir_zamir 2013-04-07, 14:48
+
Martin Kleppmann 2013-04-10, 03:55
+
nir_zamir 2013-04-10, 10:43
+
Doug Cutting 2013-04-10, 18:47
Copy link to this message
-
Re: Union in AvroMapper.map: Not in Union??
On 10 April 2013 11:47, Doug Cutting <[EMAIL PROTECTED]> wrote:

> On Tue, Apr 9, 2013 at 8:55 PM, Martin Kleppmann <[EMAIL PROTECTED]>wrote:
>
>> AvroJob does not perform schema evolution, so the schema you provide
>> to AvroJob.setInputSchema has to be the exact same schema with which
>> your input files to the mappers are encoded.
>>
>
> Actually the schema you provide is used as the "reader" schema and need
> not exactly match the schema used to write the data.  The schema used when
> writing is stored in the data files.
>

Oh yes — sorry, I misread the code. (I overlooked the fact that
DataFileStream.initialize mutates the reader to use the writer's schema.)

Martin
+
nir_zamir 2013-04-14, 08:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB