Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # dev >> Avro governance and Avro Enhancement Proposals


Copy link to this message
-
Re: Avro governance and Avro Enhancement Proposals
I'm +1 on having AEPs.  I agree we need to keep them as light-weight as
possible.  We don't want process interfering with writing code.  I like the
idea of the AEPs sitting side-by-side in subversion with the
implementations.  Jira is not a great medium for capturing standards.

We could have a single spec with well defined layers and links to the
relevant AEPs.  Initially, we might just include the AEP text but, as we
grow, we could use links/pointers instead. Even now, we have two
serialization formats (json and binary) and soon multiple RPC mechanisms.

I think it's misleading to see the spec as a single source of truth.  There
are portions of the spec which are much more set in stone (e.g. the binary
serialization format) than others (e.g. lightweight RPC) are more a work in
progress.  It would be good to make that clearer to future implementors and
users.  I think AEPs should have a status assigned to them similar to RFCs
http://en.wikipedia.org/wiki/Request_for_Comments#Status.

-Matt
On Thu, Jun 3, 2010 at 4:16 PM, Jeff Hammerbacher <[EMAIL PROTECTED]>wrote:

> My opinion roughly coincides with Philip. I don't think Avro is large
> enough
> to require AEPs just yet. I think a single spec is great, and clarification
> of levels inside that spec would be greater.
>
> On Thu, Jun 3, 2010 at 3:07 PM, Philip Zeyliger <[EMAIL PROTECTED]>
> wrote:
>
> > I'm -0: I actually really like the fact that there is one spec, and that
> > it's the source of truth.  Since Python is being brought up as an
> example,
> > I'll mention that my experience is that it's incredibly annoying to run
> > into
> > some documentation, and it points you to some proposal, and then you have
> > to
> > decipher what was proposed from what was implemented.
> >
> > I think we're still small and lightweight enough that JIRA and the
> mailing
> > list are sufficient for circulating most proposals.
> >
> > -- Philip
> >
> > On Thu, Jun 3, 2010 at 1:46 AM, Bruce Mitchener
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Hello all,
> > >
> > > I had some discussions with cutting recently about moving to an AEP
> (Avro
> > > Enhancement Proposal) process for Avro, similar to the PEP process for
> > > Python (http://www.python.org/dev/peps/).  PEP-0001 (
> > > http://www.python.org/dev/peps/pep-0001/) should give a pretty good
> (but
> > > longer) description of this process.
> > >
> > > In short, major and important changes would be put forward as an AEP.
>  An
> > > AEP would have an editor who owned the document, worked to build
> > consensus
> > > and shepherds the document to approval.  Approval of an AEP would be
> done
> > > via a vote of the PMC.
> > >
> > > AEPs could be changes to the implemenations or the processes
> surrounding
> > > Avro development (like how we format source code, how we handle code
> > > reviews, how we perform releases, etc).
> > >
> > > AEPs could be managed in the wiki or in Subversion and published to the
> > > website.  I'm interested in hearing what people think in this area.
>  I'm
> > > somewhat partial to managing them in Subversion as that:
> > >
> > >  * Puts the history in one spot.
> > >  * Allows us to update AEPs with the same patch as implementing a new
> > > feature in the code.
> > >  * Keeps things under the appropriate licensing and approval processes
> > that
> > > we have now, especially since anyone can edit the wiki.
> > >
> > > As part of this, I think that it makes sense to break some things out
> of
> > > the
> > > specification and out of their current locations on the website:
> > >
> > >  * GenAvro IDL should become a new AEP.
> > >  * An AEP should be written to describe container files and that can be
> > > removed from the spec.
> > >  * RPC mechanisms should be AEPs rather than part of the core spec.
> > >
> > > As part of this, if each AEP lists which implementations support the
> > given
> > > AEP, it would also help solve the issue being discussed in
> > > https://issues.apache.org/jira/browse/AVRO-358 (specifying levels of