Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # user - Avro versioning and SpecificDatum's


Copy link to this message
-
Re: Avro versioning and SpecificDatum's
Scott Carey 2011-09-19, 18:16
I version with SpecificDatum objects using avro data files and it works
fine.

I have seen problems arise if a user is configuring or reconfiguring the
schemas on the DatumReader passed into the construction of the
DataFileReader.
In the case of SpecificDatumReader, it is as simple as:

DatumReader<T> reader = new SpecificDatumReader<T>(T.class);
DataFileReader<T> fileReader = new DataFileReader(file, reader);

On 9/19/11 5:12 AM, "Alex Holmes" <[EMAIL PROTECTED]> wrote:

>Hi,
>
>I'm starting to play with how I can support versioning with Avro.  I
>created an initial schema, code-generated some some Java classes using
>"org.apache.avro.tool.Main compile protocol", and then used the
>DataFileWriter (with a SpecificDatumWriter) to serialize my objects to
>a file.
>
>I then modified my original schema by adding, deleting and renaming
>some fields, creating version 2 of the schema.  After re-creating the
>Java classes I attempted to read the version 1 file using the
>DataFileStream (with a SpecificDatumReader), and this is throwing an
>exception.
>
>Is versioning supported in conjunction with the SpecificDatum*
>reader/writer classes, or do I have to work at the GenericDatum level
>for this to work?
>
>Many thanks,
>Alex