-Re: Effort towards Avro 2.0?
Christophe Taton 2013-12-05, 07:40
On Wed, Dec 4, 2013 at 12:18 PM, Douglas Creager <[EMAIL PROTECTED]>wrote:
> > - unwieldy because as a user, I'll have to encode and decode the bytes
> field manually everytime I want to access this field from the original
> > record, unless I keep track of the decoded extension externally to the
> > Avro record.
> Can you handle this in the middleware? I.e., have the middleware decode
> the bytes field before passing control to the user code. That's better
> from a decoupling standpoint anyway, since the user code shouldn't care
> what middleware is wrapping it.
Well, I guess one can always handle such things externally to Avro.
I don't necessarily want to dissociate the generic part of a record and its
In some cases, the extension in isolation doesn't make sense without the
context defined by the generic part of the record.
> When you write a middleware that lets users define custom types,
> > extensions are pretty much required.
> I guess my main point is that we already have two mechanisms for dealing
> with user extensions (schema resolution and Doug's bytes field
> proposal), both of which work just fine at runtime without rebuilding or
> restarting your code. In general, I think it's better if we can solve a
> problem at the library or application level, without having to update
> the spec.
One use-case I have involves recursive records that can be extended by
Think of a tree of operations (eg. math/logic operations) that can be
extended by users with new operations.
Without support from Avro, I have to manually decode each node,
recursively, dealing with bytes at every step.
With extensions built-in, Avro could decode the entire tree in one single
Or maybe I misunderstand what you are suggesting I do instead?