Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Union in AvroMapper.map: Not in Union??


Copy link to this message
-
Re: Union in AvroMapper.map: Not in Union??
Martin Kleppmann 2013-04-10, 20:30
On 10 April 2013 11:47, Doug Cutting <[EMAIL PROTECTED]> wrote:

> On Tue, Apr 9, 2013 at 8:55 PM, Martin Kleppmann <[EMAIL PROTECTED]>wrote:
>
>> AvroJob does not perform schema evolution, so the schema you provide
>> to AvroJob.setInputSchema has to be the exact same schema with which
>> your input files to the mappers are encoded.
>>
>
> Actually the schema you provide is used as the "reader" schema and need
> not exactly match the schema used to write the data.  The schema used when
> writing is stored in the data files.
>

Oh yes — sorry, I misread the code. (I overlooked the fact that
DataFileStream.initialize mutates the reader to use the writer's schema.)

Martin