Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Avro versioning and SpecificDatum's


Copy link to this message
-
Re: Avro versioning and SpecificDatum's
What if you don't specify the schemas?

The writer schema is in the data file, and configured automatically if
unset.
The reader schema is in the class, and configured automatically in the
SpecificDatumReader constructor.

On 9/19/11 11:23 AM, "Rohini U" <[EMAIL PROTECTED]> wrote:

> I have also seen this issue when I write an avro object using
> SpecificDatumWriter and
> read it back using SpecificDatumReader, it complains saying that the schemas
> do not match even though I specify reader and writer schemas.
>
>
> On Mon, Sep 19, 2011 at 11:16 AM, Scott Carey <[EMAIL PROTECTED]> wrote:
>> I version with SpecificDatum objects using avro data files and it works
>> fine.
>>
>> I have seen problems arise if a user is configuring or reconfiguring the
>> schemas on the DatumReader passed into the construction of the
>> DataFileReader.
>>
>>
>> In the case of SpecificDatumReader, it is as simple as:
>>
>> DatumReader<T> reader = new SpecificDatumReader<T>(T.class);
>> DataFileReader<T> fileReader = new DataFileReader(file, reader);
>>
>>
>>
>> On 9/19/11 5:12 AM, "Alex Holmes" <[EMAIL PROTECTED]> wrote:
>>
>>> >Hi,
>>> >
>>> >I'm starting to play with how I can support versioning with Avro.  I
>>> >created an initial schema, code-generated some some Java classes using
>>> >"org.apache.avro.tool.Main compile protocol", and then used the
>>> >DataFileWriter (with a SpecificDatumWriter) to serialize my objects to
>>> >a file.
>>> >
>>> >I then modified my original schema by adding, deleting and renaming
>>> >some fields, creating version 2 of the schema.  After re-creating the
>>> >Java classes I attempted to read the version 1 file using the
>>> >DataFileStream (with a SpecificDatumReader), and this is throwing an
>>> >exception.
>>> >
>>> >Is versioning supported in conjunction with the SpecificDatum*
>>> >reader/writer classes, or do I have to work at the GenericDatum level
>>> >for this to work?
>>> >
>>> >Many thanks,
>>> >Alex
>>
>>
>
>
>
> --
> Regards
> -Rohini
>
> --  
>  
> People of accomplishment rarely sat back & let things happen to them. They
> went out & happened to things - Leonardo Da Vinci
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB