Please find the results of the ZooKeeper as TLP discussion here.

http://bit.ly/c4fuZT

There was consensus amongst the development team that we will stay as a
subproject of Hadoop for the time being. Full details of the discussion
can be found in the thread provided.

Patrick

Patrick Hunt wrote:
> Please find the results of the ZooKeeper as TLP discussion here.
>
> http://bit.ly/c4fuZT
>
> There was consensus amongst the development team that we will stay as a
> subproject of Hadoop for the time being. Full details of the discussion
> can be found in the thread provided.

In that discussion you state:
 > a) I do not think ZooKeeper currently has a sufficiently large and
 > diverse enough community such that it can fend for itself as a
 > TLP

How does the Hadoop PMC assist Zookeeper?  I see very little evidence
that the Hadoop PMC is involved at all day-to-day operations of
Zookeeper.  Currently the Hadoop PMC must vote on new Zookeeper
committers and releases.  These votes usually pass with the minimum
required number of PMC votes, often only after an appeal is made for
more votes.  This does not seem like significant oversight.  Beyond
that, I see little evidence of involvement by the Hadoop PMC in
Zookeeper.  So I don't see this as a strong argument not to become a TLP.

If a group of committers is operating independently, then they ought to
be an independent TLP.  The fact that you're a subproject operating
independently only hides the lack of diversity, it doesn't help it.

 > b) Loss of branding and discover-ability

This also seems a poor reason to remain a sub-project.  We can retain
prominent links to Zookeeper from hadoop.apache.org regardless of how
the projects are structured.

 > c) "if ain't broke don't fix it". I have frequent interactions with
 > Hadoop PMC/Chair and an Apache board member.

That should not change as a TLP.  Apache encourages cross-project
communication and collaboration.

The things that would change if Zookeeper were a TLP are:
  1. its official website would be at zookeeper.apache.org.
  2. it could vote for committers and releases directly rather than
through the Hadoop PMC
  3. it would submit its quarterly report to the board directly, instead
of via the Hadoop PMC.

That's pretty much it.

The board has a well established report and review system.  Each
project's quarterly report is closely read and must be individually
approved by a quorum of the board's members.  PMCs tend not to have such
a review mechanism for subprojects.  Historically subprojects that
develop problems have been slow to identify, and the problems have
worsened in the meantime.  The focus of each PMC should be on direct
decisions about code, committers and releases.  The board's job is to
make sure that each PMC operates effectively.  These are different
responsibilities and require different processes.

No one is going to force Zookeeper to become a TLP against its will, and
no change must be made immediately.  But I think such a move would be
easy to make, have significant upside for the project in simplifying its
formal votes, and have significant upside to the foundation in
facilitating the project's direct interaction with the board.

Doug

On Wed, Apr 7, 2010 at 21:00, Doug Cutting <[EMAIL PROTECTED]> wrote:
> Patrick Hunt wrote:
>>
>> Please find the results of the ZooKeeper as TLP discussion here.
>>
>> http://bit.ly/c4fuZT
>>
>> There was consensus amongst the development team that we will stay as a
>> subproject of Hadoop for the time being. Full details of the discussion can
>> be found in the thread provided.
>
> In that discussion you state:
>> a) I do not think ZooKeeper currently has a sufficiently large and
>> diverse enough community such that it can fend for itself as a
>> TLP
>
> How does the Hadoop PMC assist Zookeeper?  I see very little evidence that
> the Hadoop PMC is involved at all day-to-day operations of Zookeeper.
>  Currently the Hadoop PMC must vote on new Zookeeper committers and
> releases.  These votes usually pass with the minimum required number of PMC
> votes, often only after an appeal is made for more votes.  This does not
> seem like significant oversight.  Beyond that, I see little evidence of
> involvement by the Hadoop PMC in Zookeeper.  So I don't see this as a strong
> argument not to become a TLP.
>
> If a group of committers is operating independently, then they ought to be
> an independent TLP.  The fact that you're a subproject operating
> independently only hides the lack of diversity, it doesn't help it.
>
>> b) Loss of branding and discover-ability
>
> This also seems a poor reason to remain a sub-project.  We can retain
> prominent links to Zookeeper from hadoop.apache.org regardless of how the
> projects are structured.
>
>> c) "if ain't broke don't fix it". I have frequent interactions with
>> Hadoop PMC/Chair and an Apache board member.
>
> That should not change as a TLP.  Apache encourages cross-project
> communication and collaboration.
>
> The things that would change if Zookeeper were a TLP are:
>  1. its official website would be at zookeeper.apache.org.
>  2. it could vote for committers and releases directly rather than through
> the Hadoop PMC
>  3. it would submit its quarterly report to the board directly, instead of
> via the Hadoop PMC.
>
> That's pretty much it.
>
> The board has a well established report and review system.  Each project's
> quarterly report is closely read and must be individually approved by a
> quorum of the board's members.  PMCs tend not to have such a review
> mechanism for subprojects.  Historically subprojects that develop problems
> have been slow to identify, and the problems have worsened in the meantime.
>  The focus of each PMC should be on direct decisions about code, committers
> and releases.  The board's job is to make sure that each PMC operates
> effectively.  These are different responsibilities and require different
> processes.
>
> No one is going to force Zookeeper to become a TLP against its will, and no
> change must be made immediately.  But I think such a move would be easy to
> make, have significant upside for the project in simplifying its formal
> votes, and have significant upside to the foundation in facilitating the
> project's direct interaction with the board.
>
> Doug

+1 to all of this.

Additional comments.
If there would be 3 Zookeeper committers on the Hadoop PMC, then the
lack-of-oversight problem would go away and releases could be voted on
without delay.
I don't want to discuss why this currently is like it is - this should
really be discussed on private@hadoop.a.o.

However, going TLP requires experienced people. That means Zookeeper
will probably need Hadoop PMC members or others (ASF members, etc) to
join the initial PMC.
Otherwise, in my opinion the proper path could either be Zookeeper
growing up here for a while until a proper PMC can be forked from
Hadoop, or going through - hold your breath - incubation.

You know, this whole umbrella discussion should not result in a mass
exodus into TLPs which might collapse sooner or later.

   Bernd

Bernd Fondermann wrote:
> However, going TLP requires experienced people. That means Zookeeper
> will probably need Hadoop PMC members or others (ASF members, etc) to
> join the initial PMC.

The Hadoop PMC should use criteria similar to the Incubator for
graduation to TLP.

http://incubator.apache.org/incubation/Incubation_Policy.html#Minimum+Graduation+Requirements

The only requirement in doubt is diversity.  Those diversity
requirements are soft, not hard and fast.  When I raised this, several
board members indicated that they would rather have non-diverse TLPs
than non-diverse subprojects, since the board could then directly
observe whether diversity is improving within the project, and, if it is
not, provide assistance.

> Otherwise, in my opinion the proper path could either be Zookeeper
> growing up here for a while until a proper PMC can be forked from
> Hadoop, or going through - hold your breath - incubation.

These projects are already effectively in incubation, under the Hadoop
PMC rather than under the Incubator PMC.  The question today is whether
they're ready to graduate.  The board has asked for a report on this.

We need not rush into anything.  If we have doubts about some cases,
then our report to the board might include questions about the criteria
to promote subprojects to TLP, to help clarify things.

If we, as a PMC, wish to wait for diversity in a subproject to improve,
then we must commit to regular monitoring that subproject's diversity,
and, if it's not improving, try to figure out why, and act to help it
improve.  Just as podlings should not languish in the incubator for
years, a sub-project's goal should be promotion to TLP within a year or
so of founding.  If we don't want to commit to this oversight, then
either the board or the incubator PMC could.

Doug

On 04/08/2010 09:11 AM, Doug Cutting wrote:
> http://incubator.apache.org/incubation/Incubation_Policy.html#Minimum+Graduation+Requirements

I can speak only for ZK but we are hitting on all cylinders from this
list. All the process/infra/legal items have long ago been committed. We
have strong partnerships with multiple Apache projects (HBase and Solr
in particular) and a dedication to "the apache way".

 > The only requirement in doubt is diversity.

If you read through the full thread, and not just the snippets Doug
pulled, you would see a strong community and commitment to both
increasing diversity and gaining TLP status. The hurdle we face today
wrt diversity is that while usage is picking up, it's been historically
hard for us to gain contributors. This is primarily due to the fact that
we have a) highly targeted & complex project, and b) relatively mature
project and user base. We are not a new project, where the majority of
code is yet to be written. We are working to increase diversity, (again,
see the thread for some discussion of this) however it's slow going.
Recently we have seen a big upswing in users (twitter, digg, Solr,
Cassandra, Neo4j, etc...) due to our efforts at evangelism. The hope is
that these new users, along with some new areas for development, would
stimulate contributor growth. If you have any ideas or could provide
help in this area please feel free to discuss on our dev list.

Patrick