I'd like to start a discussion about promoting Avro from a Hadoop
sub-project to a top-level Apache project (TLP).

This is not yet a vote.  Once we have established general understanding
and agreement, I'll call a vote.

I propose we move Avro from hadoop.apache.org/avro to avro.apache.org.
Avro would then have it's own Project Management Committee (PMC) so that
it can elect committers and create releases on its own.  Currently these
actions require votes by the Hadoop PMC.  However I think Avro now has a
sufficiently large, diverse and distinct community that it can fend for
itself.

I suggest that initial Avro PMC consist of all active Avro committers at
the time we make the formal proposal.  This is typical for new TLPs.
(Subsequently PMCs tend to promote committers to the PMC.  The Hadoop
PMC generally promotes committers to the PMC after a year of consistent
activity, while some projects immediately add new committers to their
PMC.  But we don't need to decide our policy for new PMC membership now,
only the makeup of the initial PMC.)

I nominate myself as the initial chair of the Avro PMC, with the proviso
that we adopt a policy of regular chair replacement.  I suggest that
Avro PMC chairs serve a one or two-year term.  A PMC chair has no more
power than other PMC members, but rather has a few more duties.  In
particular, the chair must submit written quarterly reports to the board
describing the health of the projects developer community.  The chair
also maintains subversion permissions and committer account creation.

Do these proposal sound reasonable?  Any improvements or questions?

For some background, please read:

http://www.apache.org/foundation/how-it-works.html#roles

The steps I imagine are roughly:
  - discussion by Avro developers (what I'm starting here today)
  - vote by Avro developers
  - discussion by Hadoop PMC
  - vote by Hadoop PMC
  - draft resolution sent to board
  - board votes on resolution to form TLP

Formally, the board alone has the power to create a TLP: all prior steps
are merely an ordered means to make the case to the board that all
involved parties support such an action.

Thanks,

Doug

I like the idea of Avro as a TLP.

Avro has appeal to groups outside of the Hadoop community even though Hadoop
will likely be our biggest "customer" for some time.  Our release cycle and
decisions should be driven by this broader user community.  The fact that
Avro is a subproject of Hadoop can imply that Avro is "Hadoop RPC" or that
it "requires" Hadoop or that it's only useful for huge datasets.  I think
having Avro as a separate TLP would help disabuse people of those notions.

There is a lot of buzz around Hadoop now and leaving could have a negative
effect on our visibility.  However, I think we'll have no trouble generating
our own buzz.  We can, for example, develop strong benchmarks that show we
beat other serialization/RPC systems in performance and announce it loudly.
We already have a generous Apache license.  If we make Avro an easy drop-in
replacement for other systems, we'll likely see broad adoption.  Lots of
happy Avro consumers == buzz.

-Matt
On Wed, Mar 17, 2010 at 2:09 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> I'd like to start a discussion about promoting Avro from a Hadoop
> sub-project to a top-level Apache project (TLP).
>
> This is not yet a vote.  Once we have established general understanding and
> agreement, I'll call a vote.
>
> I propose we move Avro from hadoop.apache.org/avro to avro.apache.org.
> Avro would then have it's own Project Management Committee (PMC) so that it
> can elect committers and create releases on its own.  Currently these
> actions require votes by the Hadoop PMC.  However I think Avro now has a
> sufficiently large, diverse and distinct community that it can fend for
> itself.
>
> I suggest that initial Avro PMC consist of all active Avro committers at
> the time we make the formal proposal.  This is typical for new TLPs.
> (Subsequently PMCs tend to promote committers to the PMC.  The Hadoop PMC
> generally promotes committers to the PMC after a year of consistent
> activity, while some projects immediately add new committers to their PMC.
>  But we don't need to decide our policy for new PMC membership now, only the
> makeup of the initial PMC.)
>
> I nominate myself as the initial chair of the Avro PMC, with the proviso
> that we adopt a policy of regular chair replacement.  I suggest that Avro
> PMC chairs serve a one or two-year term.  A PMC chair has no more power than
> other PMC members, but rather has a few more duties.  In particular, the
> chair must submit written quarterly reports to the board describing the
> health of the projects developer community.  The chair also maintains
> subversion permissions and committer account creation.
>
> Do these proposal sound reasonable?  Any improvements or questions?
>
> For some background, please read:
>
> http://www.apache.org/foundation/how-it-works.html#roles
>
> The steps I imagine are roughly:
>  - discussion by Avro developers (what I'm starting here today)
>  - vote by Avro developers
>  - discussion by Hadoop PMC
>  - vote by Hadoop PMC
>  - draft resolution sent to board
>  - board votes on resolution to form TLP
>
> Formally, the board alone has the power to create a TLP: all prior steps
> are merely an ordered means to make the case to the board that all involved
> parties support such an action.
>
> Thanks,
>
> Doug
>
>
>

On Thu, Mar 18, 2010 at 9:22 AM, Matt Massie <[EMAIL PROTECTED]> wrote:
> I like the idea of Avro as a TLP.
>
> Avro has appeal to groups outside of the Hadoop community even though Hadoop
> will likely be our biggest "customer" for some time.  Our release cycle and
> decisions should be driven by this broader user community.  The fact that
> Avro is a subproject of Hadoop can imply that Avro is "Hadoop RPC" or that
> it "requires" Hadoop or that it's only useful for huge datasets.  I think
> having Avro as a separate TLP would help disabuse people of those notions.
>
> There is a lot of buzz around Hadoop now and leaving could have a negative
> effect on our visibility.  However, I think we'll have no trouble generating
> our own buzz.  We can, for example, develop strong benchmarks that show we
> beat other serialization/RPC systems in performance and announce it loudly.
> We already have a generous Apache license.  If we make Avro an easy drop-in
> replacement for other systems, we'll likely see broad adoption.  Lots of
> happy Avro consumers == buzz.

If it works well for us, I'm not worried about being able to generate buzz.

-ryan

On 3/17/10 2:09 PM, "Doug Cutting" said:

> I'd like to start a discussion about promoting Avro from a Hadoop
> sub-project to a top-level Apache project (TLP).
>

...

>
> Do these proposal sound reasonable?  Any improvements or questions?
>

Yep, many of the people I've encountered who are now using or planning to
use Avro are not doing so in a Hadoop context (they are primarily using it
for on-the-wire format), so I think it make sense as a stand-alone project.

Scott

I like the way you phrased the advantages and disadvantages of moving to a TLP.
I think its not a question of if, but a question of when this should happen.  
Should we set some goals to achieve before moving to a TLP?

Here are some ideas for such goals, some of which may already be met:
* Clearly define what a "core" feature set contains for languages.  
* "Core" functionality in at least Java, C/C++, python, and one other language.  (ProtocolBuffers +1)
* Define some sort of usability goals for the languages above.
* Performance goals and tests for the languages above.
* Improved project documentation WRT vision, wish-lists, etc.  These will often be language specific.  Perhaps a separate page for each language easy to find from the home page.

I feel that in order to take advantage of being a separate TLP, it will require most of the things above.  That does not mean we need all of the above before becoming one.  Attracting users and contributors as a TLP will require a different approach to how the project presents itself and exposes its status.  The change to a TLP also presents a unique opportunity to attract attention on its own, and when that additional attention comes, we should make sure our site, documentation, and feature set are in what we feel is a good position to take advantage of the extra attention and grow our user/contributor base.

-Scott

On Mar 18, 2010, at 9:22 AM, Matt Massie wrote:

> I like the idea of Avro as a TLP.
>
> Avro has appeal to groups outside of the Hadoop community even though Hadoop
> will likely be our biggest "customer" for some time.  Our release cycle and
> decisions should be driven by this broader user community.  The fact that
> Avro is a subproject of Hadoop can imply that Avro is "Hadoop RPC" or that
> it "requires" Hadoop or that it's only useful for huge datasets.  I think
> having Avro as a separate TLP would help disabuse people of those notions.
>
> There is a lot of buzz around Hadoop now and leaving could have a negative
> effect on our visibility.  However, I think we'll have no trouble generating
> our own buzz.  We can, for example, develop strong benchmarks that show we
> beat other serialization/RPC systems in performance and announce it loudly.
> We already have a generous Apache license.  If we make Avro an easy drop-in
> replacement for other systems, we'll likely see broad adoption.  Lots of
> happy Avro consumers == buzz.
>
> -Matt
>
>
> On Wed, Mar 17, 2010 at 2:09 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>
>> I'd like to start a discussion about promoting Avro from a Hadoop
>> sub-project to a top-level Apache project (TLP).
>>
>> This is not yet a vote.  Once we have established general understanding and
>> agreement, I'll call a vote.
>>
>> I propose we move Avro from hadoop.apache.org/avro to avro.apache.org.
>> Avro would then have it's own Project Management Committee (PMC) so that it
>> can elect committers and create releases on its own.  Currently these
>> actions require votes by the Hadoop PMC.  However I think Avro now has a
>> sufficiently large, diverse and distinct community that it can fend for
>> itself.
>>
>> I suggest that initial Avro PMC consist of all active Avro committers at
>> the time we make the formal proposal.  This is typical for new TLPs.
>> (Subsequently PMCs tend to promote committers to the PMC.  The Hadoop PMC
>> generally promotes committers to the PMC after a year of consistent
>> activity, while some projects immediately add new committers to their PMC.
>> But we don't need to decide our policy for new PMC membership now, only the
>> makeup of the initial PMC.)
>>
>> I nominate myself as the initial chair of the Avro PMC, with the proviso
>> that we adopt a policy of regular chair replacement.  I suggest that Avro
>> PMC chairs serve a one or two-year term.  A PMC chair has no more power than
>> other PMC members, but rather has a few more duties.  In particular, the
>> chair must submit written quarterly reports to the board describing the

This makes much sense for how much non-Hadoop use we see being desired
for Avro. The chair rotation is a good plan, too.
--
Jeff

On Wed, Mar 17, 2010 at 2:09 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
> I'd like to start a discussion about promoting Avro from a Hadoop
> sub-project to a top-level Apache project (TLP).
>
> This is not yet a vote.  Once we have established general understanding and
> agreement, I'll call a vote.
>
> I propose we move Avro from hadoop.apache.org/avro to avro.apache.org. Avro
> would then have it's own Project Management Committee (PMC) so that it can
> elect committers and create releases on its own.  Currently these actions
> require votes by the Hadoop PMC.  However I think Avro now has a
> sufficiently large, diverse and distinct community that it can fend for
> itself.
>
> I suggest that initial Avro PMC consist of all active Avro committers at the
> time we make the formal proposal.  This is typical for new TLPs.
> (Subsequently PMCs tend to promote committers to the PMC.  The Hadoop PMC
> generally promotes committers to the PMC after a year of consistent
> activity, while some projects immediately add new committers to their PMC.
>  But we don't need to decide our policy for new PMC membership now, only the
> makeup of the initial PMC.)
>
> I nominate myself as the initial chair of the Avro PMC, with the proviso
> that we adopt a policy of regular chair replacement.  I suggest that Avro
> PMC chairs serve a one or two-year term.  A PMC chair has no more power than
> other PMC members, but rather has a few more duties.  In particular, the
> chair must submit written quarterly reports to the board describing the
> health of the projects developer community.  The chair also maintains
> subversion permissions and committer account creation.
>
> Do these proposal sound reasonable?  Any improvements or questions?
>
> For some background, please read:
>
> http://www.apache.org/foundation/how-it-works.html#roles
>
> The steps I imagine are roughly:
>  - discussion by Avro developers (what I'm starting here today)
>  - vote by Avro developers
>  - discussion by Hadoop PMC
>  - vote by Hadoop PMC
>  - draft resolution sent to board
>  - board votes on resolution to form TLP
>
> Formally, the board alone has the power to create a TLP: all prior steps are
> merely an ordered means to make the case to the board that all involved
> parties support such an action.
>
> Thanks,
>
> Doug
>
>
>

> From: Doug Cutting [mailto:[EMAIL PROTECTED]]
> I'd like to start a discussion about promoting Avro from a Hadoop
> sub-project to a top-level Apache project (TLP).

I support the idea. It gives us some independence. I remember, for example,
we had TestNG as unit test environment for Java but moved back to JUnit only
because Hadoop will always stay with JUnit. (I'm not proposing we move to
TestNG or anything. I just gave an example).

Thiru

Doug Cutting wrote:
> I nominate myself as the initial chair of the Avro PMC

On second thought, it would be best if I were not the chair, since I
also serve on the board, and the best practice is for board members not
to chair PMCs, since the chair represents the PMC to the board.

Any volunteers for chair?  I'm happy to mentor the chair.  It'll take
only an hour a month or so, to request account creations, maintain svn
permissions, and compile a quarterly report to the board.

If no one volunteers, I'll nominate someone...

Doug

Scott Carey wrote:
> Should we set some goals to achieve before moving to a TLP?

 From Apache's viewpoint, the requirements for becoming a TLP are that
the community knows how to collaboratively write, maintain and release
software.  We have a diverse, active set of contributors, and have made
several releases, and I think we handily pass the bar.

Look for example at the Incubator graduation requirements:

http://incubator.apache.org/guides/graduation.html#requirements

Code completeness or utility are not required, however I think a good
community generally creates good code.

Doug

Doug-

I would be happy to take the responsibility of serving as PMC chair for a
year.  I'm supportive of the idea of rotating the position on a yearly
basis.  I'd love to learn more about the Apache process and hopefully
contribute in a positive way to Avro.

-Matt
On Thu, Mar 18, 2010 at 3:00 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> Doug Cutting wrote:
>
>> I nominate myself as the initial chair of the Avro PMC
>>
>
> On second thought, it would be best if I were not the chair, since I also
> serve on the board, and the best practice is for board members not to chair
> PMCs, since the chair represents the PMC to the board.
>
> Any volunteers for chair?  I'm happy to mentor the chair.  It'll take only
> an hour a month or so, to request account creations, maintain svn
> permissions, and compile a quarterly report to the board.
>
> If no one volunteers, I'll nominate someone...
>
> Doug
>

Matt Massie wrote:
> I would be happy to take the responsibility of serving as PMC chair for a
> year.

Great!  Thanks for volunteering!

Doug

+1 for Matt's nomination

On 3/18/10 3:26 PM, "Matt Massie" said:

> Doug-
>
> I would be happy to take the responsibility of serving as PMC chair for a
> year.  I'm supportive of the idea of rotating the position on a yearly
> basis.  I'd love to learn more about the Apache process and hopefully
> contribute in a positive way to Avro.
>
> -Matt
>
>
> On Thu, Mar 18, 2010 at 3:00 PM, Doug Cutting <[EMAIL PROTECTED]> wrote:
>
>> Doug Cutting wrote:
>>
>>> I nominate myself as the initial chair of the Avro PMC
>>>
>>
>> On second thought, it would be best if I were not the chair, since I also
>> serve on the board, and the best practice is for board members not to chair
>> PMCs, since the chair represents the PMC to the board.
>>
>> Any volunteers for chair?  I'm happy to mentor the chair.  It'll take only
>> an hour a month or so, to request account creations, maintain svn
>> permissions, and compile a quarterly report to the board.
>>
>> If no one volunteers, I'll nominate someone...
>>
>> Doug
>>

Doug Cutting wrote:
> I'd like to start a discussion about promoting Avro from a Hadoop
> sub-project to a top-level Apache project (TLP).

Discussion seems to have died down.  Folks generally liked the idea, and
support the nomination of Matt as initial chair.  So unless someone
objects, I'll now draft a board resolution and call a vote on it.

Thanks,

Doug