Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> Schema evolution and projection


Copy link to this message
-
Re: Schema evolution and projection
Doug,

I have updated my test code in line with your excellent example and I now
have the projection aspect working well.

Now... I'm stuck on a schema evolution test. Basically if I use your
example as the foundation and I create a new schema based on the
WRITER_SCHEMA in which I add a new field to the end (to model schema
evolution) I receive an error when trying to create the writer_iface.

writer_iface = avro_resolved_writer_new(writer_schema, reader_schema);

"Reader field Field_3 doesn't appear in writer"

Any chance you could extending your example to show the ability of Avro to
read data from a data file using an evolved schema (say in a simple
situation were a new field is added to the schema)?

Regards,
Chris

On Fri, Mar 1, 2013 at 9:08 AM, Douglas Creager <[EMAIL PROTECTED]>wrote:

> > Thanks for the informative reply. I look forward to the example code,
> > that is exactly what I'm after.
> >
> > I'm really struggling with my schema evolution testing. I thought I'd
> > post a question about schema projection because it seemed simpler but I
> > guess it also rests on creating a resolver. I have not found a clear and
> > simple example of how to do it using avro-c. I've trawled the test code
> > for examples but as I mention I can't find a clear and simple example.
>
> Alrighty, here you go:
>
> http://dcreager.github.com/avro-examples/resolved-writer.html
>
> And a git repo with the source code:
>
> https://github.com/dcreager/avro-examples/tree/master/resolved-writer
>
> I hope this helps — please let me know if you have any other questions.
>
> cheers
> –doug
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB