Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro, mail # dev - Check for schema backwards compatibility


Copy link to this message
-
Re: Check for schema backwards compatibility
Jeff Kolesky 2012-11-18, 04:04
Would it be appropriate to add this method to the Schema class itself in
the same way `subsume` and `unify` were, or would you rather see a separate
tool, similar to SchemaNormalization?

On Fri, Nov 16, 2012 at 3:54 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> On Fri, Nov 16, 2012 at 3:37 PM, Jeff Kolesky <[EMAIL PROTECTED]>
> wrote:
> > Has there been discussion of the need fot this type of tool?  Would other
> > people find it useful?
>
> I have not seen this discussed, but I can see the utility.  One could
> automatically check new schemas for compatibility with prior versions
> before using them, to ensure that both old and new data can be read
> with the new schema.  This would require checking that any added
> fields have default values specified.
>
> Related is the ability to tell if an old schema can be used to read
> data written with a newer.  This would require that any removed fields
> have a default value specified.
>
> In general, to ensure readability in both cases, one should always
> provide a default value for every field.  So a method that traversed a
> schema and verified that each field has a default value might suffice.
>
> Doug
>