Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Check for schema backwards compatibility


Copy link to this message
-
Re: Check for schema backwards compatibility
On Fri, Nov 16, 2012 at 3:37 PM, Jeff Kolesky <[EMAIL PROTECTED]> wrote:
> Has there been discussion of the need fot this type of tool?  Would other
> people find it useful?

I have not seen this discussed, but I can see the utility.  One could
automatically check new schemas for compatibility with prior versions
before using them, to ensure that both old and new data can be read
with the new schema.  This would require checking that any added
fields have default values specified.

Related is the ability to tell if an old schema can be used to read
data written with a newer.  This would require that any removed fields
have a default value specified.

In general, to ensure readability in both cases, one should always
provide a default value for every field.  So a method that traversed a
schema and verified that each field has a default value might suffice.

Doug
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB