Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro versioning and SpecificDatum's


Copy link to this message
-
Re: Avro versioning and SpecificDatum's
I version with SpecificDatum objects using avro data files and it works
fine.

I have seen problems arise if a user is configuring or reconfiguring the
schemas on the DatumReader passed into the construction of the
DataFileReader.
In the case of SpecificDatumReader, it is as simple as:

DatumReader<T> reader = new SpecificDatumReader<T>(T.class);
DataFileReader<T> fileReader = new DataFileReader(file, reader);

On 9/19/11 5:12 AM, "Alex Holmes" <[EMAIL PROTECTED]> wrote:

>Hi,
>
>I'm starting to play with how I can support versioning with Avro.  I
>created an initial schema, code-generated some some Java classes using
>"org.apache.avro.tool.Main compile protocol", and then used the
>DataFileWriter (with a SpecificDatumWriter) to serialize my objects to
>a file.
>
>I then modified my original schema by adding, deleting and renaming
>some fields, creating version 2 of the schema.  After re-creating the
>Java classes I attempted to read the version 1 file using the
>DataFileStream (with a SpecificDatumReader), and this is throwing an
>exception.
>
>Is versioning supported in conjunction with the SpecificDatum*
>reader/writer classes, or do I have to work at the GenericDatum level
>for this to work?
>
>Many thanks,
>Alex
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB