Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro >> mail # user >> Union in AvroMapper.map: Not in Union??


+
nir_zamir 2013-03-24, 10:23
+
nir_zamir 2013-04-07, 14:48
+
Martin Kleppmann 2013-04-10, 03:55
+
nir_zamir 2013-04-10, 10:43
+
Doug Cutting 2013-04-10, 18:47
Copy link to this message
-
Re: Union in AvroMapper.map: Not in Union??
On 10 April 2013 11:47, Doug Cutting <[EMAIL PROTECTED]> wrote:

> On Tue, Apr 9, 2013 at 8:55 PM, Martin Kleppmann <[EMAIL PROTECTED]>wrote:
>
>> AvroJob does not perform schema evolution, so the schema you provide
>> to AvroJob.setInputSchema has to be the exact same schema with which
>> your input files to the mappers are encoded.
>>
>
> Actually the schema you provide is used as the "reader" schema and need
> not exactly match the schema used to write the data.  The schema used when
> writing is stored in the data files.
>

Oh yes — sorry, I misread the code. (I overlooked the fact that
DataFileStream.initialize mutates the reader to use the writer's schema.)

Martin
+
nir_zamir 2013-04-14, 08:39