Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Avro, mail # user - Properties on union schema?


+
Christophe Taton 2013-07-14, 02:01
+
Connor Doyle 2013-07-14, 02:15
+
Christophe Taton 2013-07-15, 17:57
+
Doug Cutting 2013-07-15, 18:25
+
Christophe Taton 2013-07-16, 07:40
+
Doug Cutting 2013-07-16, 18:13
Copy link to this message
-
Re: Properties on union schema?
Christophe Taton 2013-07-18, 17:23
Hi Doug,

I realize I have strong interests in several issues that would imply
forward-incompatible changes (eg. AVRO-248 and AVRO-530).
Do you believe it is possible to introduce part of these as experimental
features that would be disabled by default, or would you rather isolate
such changes in completely separate releases? I would happily invest energy
in either approach.

Thanks,
C.
On Tue, Jul 16, 2013 at 11:13 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> We've promised to maintain forward and backward compatibility until
> version 2.0.  There have been a number of incompatible schema changes
> requested over the history of Avro, but none have yet been made.  If
> we ever decide to make any, we should perhaps try to make them all at
> once and provide good conversion tools.
>
> Doug
>
> On Tue, Jul 16, 2013 at 12:40 AM, Christophe Taton <[EMAIL PROTECTED]>
> wrote:
> > Hi Doug,
> >
> > Thanks for the explanation.
> > How far back is it reasonable to maintain backward and/or forward
> > compatibility?
> >
> > Assuming there is a limit to the forward compatibility requirement,
> could we
> > introduce the ability to read schemas with extended union descriptors,
> > without the ability to write such descriptors, and introduce the ability
> to
> > write after enough releases have passed?
> >
> > Thanks,
> > C.
> >
> >
> > On Mon, Jul 15, 2013 at 11:25 AM, Doug Cutting <[EMAIL PROTECTED]>
> wrote:
> >>
> >> The reason is simply that the JSON syntax for union schemas doesn't
> >> permit properties.  To support properties we'd need to add an
> >> alternate syntax for unions, but I don't see how to do that
> >> compatibly, so that an old client seeing one of the new union schemas
> >> would still be able to process the data.
> >>
> >> For example, the syntax might be something like:
> >>
> >> {"type":"union", "branches":[...], "prop1":"val1", ...}
> >>
> >> But that would cause errors in every existing implementation.
> >>
> >> Doug
> >>
> >> On Sat, Jul 13, 2013 at 7:01 PM, Christophe Taton <[EMAIL PROTECTED]>
> >> wrote:
> >> > Hi,
> >> > I'm toying with a few changes to provide alternative representations
> of
> >> > union fields in Java (somewhat related to
> >> > https://issues.apache.org/jira/browse/AVRO-248).
> >> > To experiment with this, I'd like to set properties on union schemas,
> >> > but
> >> > properties are currently disabled on unions.
> >> > Is there a particular reason for this, or is it a reasonable change to
> >> > allow
> >> > properties on unions?
> >> > Thanks,
> >> > C.
> >
> >
>
+
Doug Cutting 2013-07-18, 18:29