Eric and the team,

First, let me provide a little background about us. We at Intel have been using Chukwa for building HiTune (a Hadoop performance analyzer https://github.com/intel-hadoop/hitune), and one of our key team member, Jie Huang, was recently accepted as a Chukwa committer (unfortunately she was out sick since late September and has not been as active in the Chukwa community as we would like).

IMO, a key question for the Chukwa project is on how to grow the community, and I believe an active developer community is driven by active users.  It is unclear to me at this moment who are using Chukwa in their daily work, what it is being used for, and how it can play an important role in its target domain. I would suggest people on the list to share their usage as the first step - How are you using Chukwa? Do you think Chukwa is a good solution that can attract new users for that specific problem?

As a starter, I'll share our usage:
1)      We have been using Chukwa to collect and aggregate performance metric from Hadoop cluster, so that our tool HiTune can analyze performance of Hadoop applications.
2)      And as we outlined in CHUKWA-665, we have a prototype that uses Chukwa to collect and aggregate cluster system metrics, which powers the Ganglia web frontend for cluster monitoring.

IMHO, at this moment Flume is winning mindshare for distributed data collection (e.g., ETL), and Ganglia & Nagios are the cluster monitoring of choice; I wonder what your takes are on how Chukwa can differentiate in these domains, or maybe there are some other domains Chukwa is good at.

Thanks,
-Jason

-----Original Message-----
From: Eric Yang [mailto:[EMAIL PROTECTED]]<mailto:[mailto:[EMAIL PROTECTED]]>
Sent: Mon, 26 Nov 2012 03:33:12 GMT
Subject: What constitute a successful project?

Hi IPMC,

For the past two years, Chukwa has been labelled as non-active project by
mentors, and has been put on votes for retiring this project by mentor and
IPMC.
In this year's stats, Chukwa has more activities in comparison to Apache
Wink in both mailing list traffic and resolved jiras.  Yet Chukwa has been
voted to discontinue by mentors, but Wink is voted to graduate  by the same
mentor. Here are the number of mails showed up in dev list between Apache
Chukwa and Apache Wink:

...

Hi Jason,

IBM is using Chukwa agent as the base of monitoring component for
BigInsights.  The monitoring system share the same design principal, but
has been custom built for BigInsights.  We wrote some generic adaptors to
collect data from SNMP, JMX, and REST, which we are currently seeking
approval from IBM to contribute back to open source.   BigInsights is IBM's
distribution of Apache Hadoop.  We use it to monitor Hadoop and related
technologies, and Chukwa is reliable and works well for us.

Being able to have raw time series metrics and logs correlate events
together.  Chukwa approach is definitely better than plain Ganglia and
Nagios.  In Nagios and Ganglia combination, you only get facts after
irreversible events have happened.  Such as jobtracker stop responding, or
HBase region server died.  With raw data collected and analyzed, we can
prevent irreversible events from happening.  For example, problematic job
can be terminated before the job grow out of control.

Netflix has a number of presentation talking about how they use Chukwa to
stream data to EC2.  The most recent presentation is here:
http://cdn.oreillystatic.com/en/assets/1/event/85/Netflix_s%20Evolving%20Data%20Science%20Architecture%20Presentation.pdf

regards,
Eric

On Thu, Nov 29, 2012 at 5:54 AM, Dai, Jason <[EMAIL PROTECTED]> wrote:

> Eric and the team,
>
> First, let me provide a little background about us. We at Intel have been
> using Chukwa for building HiTune (a Hadoop performance analyzer
> https://github.com/intel-hadoop/hitune), and one of our key team member,
> Jie Huang, was recently accepted as a Chukwa committer (unfortunately she
> was out sick since late September and has not been as active in the Chukwa
> community as we would like).
>
> IMO, a key question for the Chukwa project is on how to grow the
> community, and I believe an active developer community is driven by active
> users.  It is unclear to me at this moment who are using Chukwa in their
> daily work, what it is being used for, and how it can play an important
> role in its target domain. I would suggest people on the list to share
> their usage as the first step - How are you using Chukwa? Do you think
> Chukwa is a good solution that can attract new users for that specific
> problem?
>
> As a starter, I'll share our usage:
> 1)      We have been using Chukwa to collect and aggregate performance
> metric from Hadoop cluster, so that our tool HiTune can analyze performance
> of Hadoop applications.
> 2)      And as we outlined in CHUKWA-665, we have a prototype that uses
> Chukwa to collect and aggregate cluster system metrics, which powers the
> Ganglia web frontend for cluster monitoring.
>
> IMHO, at this moment Flume is winning mindshare for distributed data
> collection (e.g., ETL), and Ganglia & Nagios are the cluster monitoring of
> choice; I wonder what your takes are on how Chukwa can differentiate in
> these domains, or maybe there are some other domains Chukwa is good at.
>
> Thanks,
> -Jason
>
> -----Original Message-----
> From: Eric Yang [mailto:[EMAIL PROTECTED]]<mailto:[mailto:
> [EMAIL PROTECTED]]>
> Sent: Mon, 26 Nov 2012 03:33:12 GMT
> Subject: What constitute a successful project?
>
> Hi IPMC,
>
> For the past two years, Chukwa has been labelled as non-active project by
> mentors, and has been put on votes for retiring this project by mentor and
> IPMC.
> In this year's stats, Chukwa has more activities in comparison to Apache
> Wink in both mailing list traffic and resolved jiras.  Yet Chukwa has been
> voted to discontinue by mentors, but Wink is voted to graduate  by the same
> mentor. Here are the number of mails showed up in dev list between Apache
> Chukwa and Apache Wink:
>
> ...
>
>
>

Hi Eric,
Sorry to interfere at that point but I cannot let you using my name and
Netflix together for Chukwa.
I've designed Chukwa and I'm the main architect behind Chukwa, correct.

However Netflix is NOT running CHUKWA but HONU.
Honu is a stream based data collection that run at scale at Netflix and
other places.
I've designed Honu when I was at Netflix and Honu does not use CHUKWA code
anymore.
Honu code is a complete rewrite done by me and only me and that's the
reason why Honu scale
to more than 60 billions events/day.
People are still using the name Chukwa because it was the name I used for
my first presentation.
I've changed the name to Honu when I started the complete rewrite and you
are aware of that.

I'm the architect of both so there's some similarities but
Chukwa will never be Honu and I cannot let people think that they are.

I'll ask Kurt to update his presentation and to use the correct name: HONU
and not CHUKWA.

You can read more about Honu:
Here: http://www.slideshare.net/jboulon/hadoop-summit-2010-honu
or here:
http://www.slideshare.net/jboulon/cloud-connect-2012-big-data-netflix

Sorry Eric and next time you use my work please verify your sources or I'll
have to take
a more active role.

/Jerome Boulon
[EMAIL PROTECTED]
On Thu, Nov 29, 2012 at 10:39 PM, Eric Yang <[EMAIL PROTECTED]> wrote:

> Hi Jason,
>
> IBM is using Chukwa agent as the base of monitoring component for
> BigInsights.  The monitoring system share the same design principal, but
> has been custom built for BigInsights.  We wrote some generic adaptors to
> collect data from SNMP, JMX, and REST, which we are currently seeking
> approval from IBM to contribute back to open source.   BigInsights is IBM's
> distribution of Apache Hadoop.  We use it to monitor Hadoop and related
> technologies, and Chukwa is reliable and works well for us.
>
> Being able to have raw time series metrics and logs correlate events
> together.  Chukwa approach is definitely better than plain Ganglia and
> Nagios.  In Nagios and Ganglia combination, you only get facts after
> irreversible events have happened.  Such as jobtracker stop responding, or
> HBase region server died.  With raw data collected and analyzed, we can
> prevent irreversible events from happening.  For example, problematic job
> can be terminated before the job grow out of control.
>
> Netflix has a number of presentation talking about how they use Chukwa to
> stream data to EC2.  The most recent presentation is here:
>
> http://cdn.oreillystatic.com/en/assets/1/event/85/Netflix_s%20Evolving%20Data%20Science%20Architecture%20Presentation.pdf
>
> regards,
> Eric
>
> On Thu, Nov 29, 2012 at 5:54 AM, Dai, Jason <[EMAIL PROTECTED]> wrote:
>
> > Eric and the team,
> >
> > First, let me provide a little background about us. We at Intel have been
> > using Chukwa for building HiTune (a Hadoop performance analyzer
> > https://github.com/intel-hadoop/hitune), and one of our key team member,
> > Jie Huang, was recently accepted as a Chukwa committer (unfortunately she
> > was out sick since late September and has not been as active in the
> Chukwa
> > community as we would like).
> >
> > IMO, a key question for the Chukwa project is on how to grow the
> > community, and I believe an active developer community is driven by
> active
> > users.  It is unclear to me at this moment who are using Chukwa in their
> > daily work, what it is being used for, and how it can play an important
> > role in its target domain. I would suggest people on the list to share
> > their usage as the first step - How are you using Chukwa? Do you think
> > Chukwa is a good solution that can attract new users for that specific
> > problem?
> >
> > As a starter, I'll share our usage:
> > 1)      We have been using Chukwa to collect and aggregate performance
> > metric from Hadoop cluster, so that our tool HiTune can analyze
> performance
> > of Hadoop applications.
> > 2)      And as we outlined in CHUKWA-665, we have a prototype that uses

/Jerome

On Nov 30, 2012, at 1:04 AM, Jerome Boulon wrote:

> Sorry Eric and next time you use my work please verify your sources or I'll have to take a more active role.

I can see why Eric, and others, became confused by the slides that were presented by Netflix last month.  Thank you for your kind update.  
Regards,
Alan

Apache is a non-profit organization.  If we restrict our thinking model to
metrics of how many developers, and how many patches are committed in
pre-defeined time limit.  There is no software that is gong to succeed in
this evaluation other than commercial software.  Paid developers are
contributing to the software that meeting cooperate interests at rapid
pace, and smaller companies will work together until cooperate interests
tear apart the software, or the funding eventually dry up and the software
cease to exist, and the community will eventually fall apart.  Good
software usually comes down to a few individuals who work hard to enable
the community to flourish.  Many of the good software takes decades to
develop from hobby projects.  I will accept the voting result from IPMC,
and I wish IPMC would use better human sense to enable future project to
flourish.

Chris Douglas resigned from mentor position, therefore, Chukwa will need a
new mentor, and one of Chukwa contributor Sourygna Luangsay volunteer to be
the motivator for Chukwa development if Chukwa is voted to stay for another
6 months.

regards,
Eric

On Mon, Nov 26, 2012 at 10:20 PM, Bernd Fondermann <
[EMAIL PROTECTED]> wrote:

> On Tue, Nov 27, 2012 at 1:25 AM, Jukka Zitting <[EMAIL PROTECTED]>
> wrote:
> > Hi,
> >
> > On Mon, Nov 26, 2012 at 10:53 PM, Alan Cabrera <[EMAIL PROTECTED]>
> wrote:
> >> As I mentioned in an earlier email, we did have this conversation seven
> >> months ago.  We came to a consensus to give it another try.  We even
> added
> >> a few committers a "bit early" with the hopes that they would infuse
> the project
> >> with more energy.
> >
> > That doesn't take away the fact that there are still people who are
> > clearly interested in continuing work on the project. Instead of
> > telling the community to pick up their toys and leave, I'd much rather
> > ask them to come up with a credible alternative. The failure of past
> > attempts to grow the community does not necessarily mean that future
> > attempts will also fail, so I'd give the community the benefit of
> > doubt as long as there are new ideas and people willing to try them.
> >
> > If I understand correctly the problems in Chukwa are two-fold: 1) the
> > community isn't diverse, i.e. there are only few people involved, and
> > 2) the community isn't active, in that even the involved people don't
> > have too many cycles to spend on the project.
> >
> > Thus I'd raise the following questions to Eric and others who want to
> > keep Chukwa alive at the ASF:
> >
> > a) Is it reasonable to expect existing community members to become
> > more active in near future? If yes, will such increased activity be
> > sustainable over a longer period of time?  Why? IIUC there was some
> > recent legal progress that might help here. What would be the best way
> > to measure the expected increase in activity?
> >
> > b) How do you expect to get more people involved in the project? What
> > concrete actions will be taken to increase the chances of new
> > contributors showing up? Why do you believe these things will work
> > better than the mentioned earlier attempts at growing the community?
> > Good ideas of concrete actions are for example cutting new releases,
> > improving project documentation, presenting the project at various
> > venues, simplifying the project build and initial setup, and giving
> > more timely answers and feedback to new users and contributors (see
> > also my observation from October [1]). How can we best tell whether
> > such efforts are working?
> >
> > Coming up with good answers to such questions is not necessarily easy
> > (and it's fine if not all of them can yet be answered), but going
> > through that effort should give us a good reason to continue the
> > incubation of Chukwa at least for a few more months until we should
> > start seeing some concrete and sustainable improvements in community
> > activity and diversity.
>
> This is exactly what we did for the last months (years, actually).

On Nov 27, 2012, at 12:08 AM, Eric Yang wrote:

> Apache is a non-profit organization.  If we restrict our thinking model to
> metrics of how many developers, and how many patches are committed in
> pre-defeined time limit.  There is no software that is gong to succeed in
> this evaluation other than commercial software.  Paid developers are
> contributing to the software that meeting cooperate interests at rapid
> pace, and smaller companies will work together until cooperate interests
> tear apart the software, or the funding eventually dry up and the software
> cease to exist, and the community will eventually fall apart.  Good
> software usually comes down to a few individuals who work hard to enable
> the community to flourish.  Many of the good software takes decades to
> develop from hobby projects.  I will accept the voting result from IPMC,
> and I wish IPMC would use better human sense to enable future project to
> flourish.

The ASF is not about code.  It's about community.  You cannot have a community of one.  There are many high quality software projects that are being developed by lone coders.  You'll find them on GitHub, SourceForge, etc.

> Chris Douglas resigned from mentor position, therefore, Chukwa will need a
> new mentor, and one of Chukwa contributor Sourygna Luangsay volunteer to be
> the motivator for Chukwa development if Chukwa is voted to stay for another
> 6 months.
Sourygna needs to step up and volunteer on his own.  But, to my mind, we have more than enough cheerleaders.  We need coders.  JMHO.
Regards,
Alan

On Nov 27, 2012, at 3:08 AM, Eric Yang <[EMAIL PROTECTED]> wrote:

> Apache is a non-profit organization.  If we restrict our thinking model to
> metrics of how many developers, and how many patches are committed in
> pre-defeined time limit.  There is no software that is gong to succeed in
> this evaluation other than commercial software.  Paid developers are
> contributing to the software that meeting cooperate interests at rapid
> pace, and smaller companies will work together until cooperate interests
> tear apart the software, or the funding eventually dry up and the software
> cease to exist, and the community will eventually fall apart.  Good
> software usually comes down to a few individuals who work hard to enable
> the community to flourish.  Many of the good software takes decades to
> develop from hobby projects.  I will accept the voting result from IPMC,
> and I wish IPMC would use better human sense to enable future project to
> flourish.

Hi Eric,

Its good to see Jukka and Ant stepping up as mentors, may be that will give you Chukwa one more chance. From browsing through the private list and the general list, I see lots of philosophical arguments and how you will bring in your patches now that legal review at your employer is over. Ofcourse you mention new volunteers too. But so far I haven't seen an answer from you or other Chukwa PPMC "what have you done previously to grow the community, what did not work and what is the change in plan now"? I see multiple variants of this question has been asked quite a few times in the last couple of days and I am eager to see an answer from the Chukwa PPMC.

Suresh
> Chris Douglas resigned from mentor position, therefore, Chukwa will need a
> new mentor, and one of Chukwa contributor Sourygna Luangsay volunteer to be
> the motivator for Chukwa development if Chukwa is voted to stay for another
> 6 months.
>
> regards,
> Eric
>
> On Mon, Nov 26, 2012 at 10:20 PM, Bernd Fondermann <
> [EMAIL PROTECTED]> wrote:
>
>> On Tue, Nov 27, 2012 at 1:25 AM, Jukka Zitting <[EMAIL PROTECTED]>
>> wrote:
>>> Hi,
>>>
>>> On Mon, Nov 26, 2012 at 10:53 PM, Alan Cabrera <[EMAIL PROTECTED]>
>> wrote:
>>>> As I mentioned in an earlier email, we did have this conversation seven
>>>> months ago.  We came to a consensus to give it another try.  We even
>> added
>>>> a few committers a "bit early" with the hopes that they would infuse
>> the project
>>>> with more energy.
>>>
>>> That doesn't take away the fact that there are still people who are
>>> clearly interested in continuing work on the project. Instead of
>>> telling the community to pick up their toys and leave, I'd much rather
>>> ask them to come up with a credible alternative. The failure of past
>>> attempts to grow the community does not necessarily mean that future
>>> attempts will also fail, so I'd give the community the benefit of
>>> doubt as long as there are new ideas and people willing to try them.
>>>
>>> If I understand correctly the problems in Chukwa are two-fold: 1) the
>>> community isn't diverse, i.e. there are only few people involved, and
>>> 2) the community isn't active, in that even the involved people don't
>>> have too many cycles to spend on the project.
>>>
>>> Thus I'd raise the following questions to Eric and others who want to
>>> keep Chukwa alive at the ASF:
>>>
>>> a) Is it reasonable to expect existing community members to become
>>> more active in near future? If yes, will such increased activity be
>>> sustainable over a longer period of time?  Why? IIUC there was some
>>> recent legal progress that might help here. What would be the best way
>>> to measure the expected increase in activity?
>>>
>>> b) How do you expect to get more people involved in the project? What
>>> concrete actions will be taken to increase the chances of new
>>> contributors showing up? Why do you believe these things will work
>>> better than the mentioned earlier attempts at growing the community?

As chair of the IPMC, I do not think that it is appropriate to have a
vote to continue incubation for six months, with no consideration of
success in between. I think that it would be reasonable to put aside
the vote to retire, and expect a plan, with contributions from more
than one non-mentor, in the next month, and some progress after that.
I also think it would be within the mission and discretion of the
committee to go ahead and vote to retire.

If it's really true that recently resolved legal muddles have been the
one barrier to success, then the removal of that barrier should
unleash some fairly substantial results.

To address the more philosophical discussion here:

The incubator is a structure set up to bootstrap communities. It's not
the only possible structure of this kind, and it's not necessarily the
best one. Like everything else at a *volunteer* organization, it is
constrained by the amount of volunteer labor available. In a perfect
world, yes, the Foundation might operate a sort of
home-for-small-projects. Such a structure would allow arbitrarily
small projects to benefit from Foundation infrastructure and legal
benefits.

However, this isn't a perfect world, and we are indeed very
constrained by the volunteer labor, and so we aren't providing a home
for years on end. There are other ways for this project to succeed
other than as an Apache TLP. You could find a related, existing,
project, and merge into them. You could set up shop on github.

On Tue, Nov 27, 2012 at 7:36 AM, Suresh Marru <[EMAIL PROTECTED]> wrote:
> On Nov 27, 2012, at 3:08 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
>
>> Apache is a non-profit organization.  If we restrict our thinking model to
>> metrics of how many developers, and how many patches are committed in
>> pre-defeined time limit.  There is no software that is gong to succeed in
>> this evaluation other than commercial software.  Paid developers are
>> contributing to the software that meeting cooperate interests at rapid
>> pace, and smaller companies will work together until cooperate interests
>> tear apart the software, or the funding eventually dry up and the software
>> cease to exist, and the community will eventually fall apart.  Good
>> software usually comes down to a few individuals who work hard to enable
>> the community to flourish.  Many of the good software takes decades to
>> develop from hobby projects.  I will accept the voting result from IPMC,
>> and I wish IPMC would use better human sense to enable future project to
>> flourish.
>
> Hi Eric,
>
> Its good to see Jukka and Ant stepping up as mentors, may be that will give you Chukwa one more chance. From browsing through the private list and the general list, I see lots of philosophical arguments and how you will bring in your patches now that legal review at your employer is over. Ofcourse you mention new volunteers too. But so far I haven't seen an answer from you or other Chukwa PPMC "what have you done previously to grow the community, what did not work and what is the change in plan now"? I see multiple variants of this question has been asked quite a few times in the last couple of days and I am eager to see an answer from the Chukwa PPMC.
>
> Suresh
>
>
>> Chris Douglas resigned from mentor position, therefore, Chukwa will need a
>> new mentor, and one of Chukwa contributor Sourygna Luangsay volunteer to be
>> the motivator for Chukwa development if Chukwa is voted to stay for another
>> 6 months.
>>
>> regards,
>> Eric
>>
>> On Mon, Nov 26, 2012 at 10:20 PM, Bernd Fondermann <
>> [EMAIL PROTECTED]> wrote:
>>
>>> On Tue, Nov 27, 2012 at 1:25 AM, Jukka Zitting <[EMAIL PROTECTED]>
>>> wrote:
>>>> Hi,
>>>>
>>>> On Mon, Nov 26, 2012 at 10:53 PM, Alan Cabrera <[EMAIL PROTECTED]>
>>> wrote:
>>>>> As I mentioned in an earlier email, we did have this conversation seven
>>>>> months ago.  We came to a consensus to give it another try.  We even
>>> added
>>>>> a few committers a "bit early" with the hopes that they would infuse

Unless there are compelling reason to stop, i.e continuing breaches of
basic ASF polices and principles, then where possible letting a poddling
continue incubation or just graduate seems better to me than making them go
elsewhere. Its not like a small slow problem is chewing up ASF resources,
but i understand not everyone here agrees with my views on that. Wink is an
example of poddling in similar circumstances and there we are about to have
decided that graduation is better than retirement. Perhaps thats a better
approach. I don't recall a graduation recommendation request from the
Incubator has ever been rejected by the board so perhaps the Incubator is
too conservative with graduation recommendations.

Its interesting comparing Wink and Chukwa. From many perspectives Chukwa is
much more active than Wink but we're about to graduate Wink and talking
about retiring this one. I've not yet had a chance to go through all the
Chukwa archives but unless i'm misunderstanding something Chukwa isn't just
a lone coder, there have been several committers in the last months and
while one is doing the majority of the commits many of those are actually
applying patches from other people, so it looks like there are a bunch of
people out there working on the project and we need to find ways of better
integrating them into the poddling community.

   ...ant

On Tue, Nov 27, 2012 at 1:54 PM, Benson Margulies <[EMAIL PROTECTED]>wrote:

> As chair of the IPMC, I do not think that it is appropriate to have a
> vote to continue incubation for six months, with no consideration of
> success in between. I think that it would be reasonable to put aside
> the vote to retire, and expect a plan, with contributions from more
> than one non-mentor, in the next month, and some progress after that.
> I also think it would be within the mission and discretion of the
> committee to go ahead and vote to retire.
>
> If it's really true that recently resolved legal muddles have been the
> one barrier to success, then the removal of that barrier should
> unleash some fairly substantial results.
>
> To address the more philosophical discussion here:
>
> The incubator is a structure set up to bootstrap communities. It's not
> the only possible structure of this kind, and it's not necessarily the
> best one. Like everything else at a *volunteer* organization, it is
> constrained by the amount of volunteer labor available. In a perfect
> world, yes, the Foundation might operate a sort of
> home-for-small-projects. Such a structure would allow arbitrarily
> small projects to benefit from Foundation infrastructure and legal
> benefits.
>
> However, this isn't a perfect world, and we are indeed very
> constrained by the volunteer labor, and so we aren't providing a home
> for years on end. There are other ways for this project to succeed
> other than as an Apache TLP. You could find a related, existing,
> project, and merge into them. You could set up shop on github.
>
>
>
> On Tue, Nov 27, 2012 at 7:36 AM, Suresh Marru <[EMAIL PROTECTED]> wrote:
> > On Nov 27, 2012, at 3:08 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
> >
> >> Apache is a non-profit organization.  If we restrict our thinking model
> to
> >> metrics of how many developers, and how many patches are committed in
> >> pre-defeined time limit.  There is no software that is gong to succeed
> in
> >> this evaluation other than commercial software.  Paid developers are
> >> contributing to the software that meeting cooperate interests at rapid
> >> pace, and smaller companies will work together until cooperate interests
> >> tear apart the software, or the funding eventually dry up and the
> software
> >> cease to exist, and the community will eventually fall apart.  Good
> >> software usually comes down to a few individuals who work hard to enable
> >> the community to flourish.  Many of the good software takes decades to
> >> develop from hobby projects.  I will accept the voting result from IPMC,
> >> and I wish IPMC would use better human sense to enable future project to

On Nov 27, 2012, at 7:16 AM, ant elder wrote:

> Unless there are compelling reason to stop, i.e continuing breaches of
> basic ASF polices and principles, then where possible letting a poddling
> continue incubation or just graduate seems better to me than making them go
> elsewhere. Its not like a small slow problem is chewing up ASF resources,
> but i understand not everyone here agrees with my views on that. Wink is an
> example of poddling in similar circumstances and there we are about to have
> decided that graduation is better than retirement. Perhaps thats a better
> approach. I don't recall a graduation recommendation request from the
> Incubator has ever been rejected by the board so perhaps the Incubator is
> too conservative with graduation recommendations.
>
> Its interesting comparing Wink and Chukwa. From many perspectives Chukwa is
> much more active than Wink but we're about to graduate Wink and talking
> about retiring this one. I've not yet had a chance to go through all the
> Chukwa archives but unless i'm misunderstanding something Chukwa isn't just
> a lone coder, there have been several committers in the last months and
> while one is doing the majority of the commits many of those are actually
> applying patches from other people, so it looks like there are a bunch of
> people out there working on the project and we need to find ways of better
> integrating them into the poddling community.

This is an interesting line of reasoning worth pursuing, IMO.  If Chukwa and Wink are actually on a par with each other we should see if it make sense to apply the same reasoning about Wink to Chukwa.

Are we implicitly having a policy change with podlings, if so, should we make it explicit?
Regards,
Alan

Hi Alan,

In Wink, you voted +1, and in Chukwa, you voted -1.  While the status are
similar between Chukwa and Wink, but what is the logic behind your votes?
 In addition, Chukwa and Kafka are similar, and some Kafka design are
borrowed from Chukwa.  Does your relationship with Kafka influence your
judgement being bias toward Chukwa?

You called me a lone developer, while the jira list showed there are
several others contributing as well.  There are people submitting patches
and open jira for discussions.

You volunteered to work as mentor for Chukwa, but we only hear from you 4
times while being Chukwa mentor:

1. March 23, we welcome you to become Chukwa mentor.  On the same day, you
ask Chukwa to be retired.
2. June Report, Chris reviewed report, you gave a +1.
3. September 9, you said "thank you" to Bernd for follow up on granting new
committers access.
4. Nov 16, you start on the private list on the same thread about retiring
Chukwa.

I am sorry, but I may be blunt.  I think your action is harmful to Chukwa
community rather than helping the community.

In the Chukwa private mailing list, I also expressed my limitation to be
contributing to Chukwa while I am working through logistics with my
employer to get approval.  While I did not write new code for Chukwa for
the past half year, Chukwa continue to receive patches from the following
individuals:

Noel Duffy
Jie Huang
Sourygna Luangsay
Abhijit Dhar
Saisai Shao
Ivy Tang
Prakhar Srivastava
Eric Speck

Some patches are committed by Ari Rabkin.  Contributions after 0.5 release
can be tracked in CHANGES.txt.  Chukwa is truly running as an open Apache
project where patches are reviewed and discussed.

Chukwa is used by Netflix, IBM, Intel, and several companies.  If you
search on LinkedIn, number of people that has Chukwa on their resume grown
from 30 people in January to 55 now.  The information are the same
information that I provided on chukwa-private mailing list.

Here is the quote from Chris:

"Eric, you recommend graduating Chukwa to be a TLP based on its
activity relative to other incubator projects, engagement from
independent contributors, adoption and investment in commercial
offerings, and indirect measures of a growing interest in the project.
Is that a fair summary?"

While Chukwa community is low key on participate political votes because
the same rehash of closing the community has been on the focus since the
original incubation proposal was written.

If we are going to move forward, more time in incubation is not a realistic
option.  The only way is vote for graduation and avoid the vicious cycle of
closing the project review.

regards,
Eric

On Tue, Nov 27, 2012 at 7:32 AM, Alan Cabrera <[EMAIL PROTECTED]> wrote:

>
> On Nov 27, 2012, at 7:16 AM, ant elder wrote:
>
> > Unless there are compelling reason to stop, i.e continuing breaches of
> > basic ASF polices and principles, then where possible letting a poddling
> > continue incubation or just graduate seems better to me than making them
> go
> > elsewhere. Its not like a small slow problem is chewing up ASF resources,
> > but i understand not everyone here agrees with my views on that. Wink is
> an
> > example of poddling in similar circumstances and there we are about to
> have
> > decided that graduation is better than retirement. Perhaps thats a better
> > approach. I don't recall a graduation recommendation request from the
> > Incubator has ever been rejected by the board so perhaps the Incubator is
> > too conservative with graduation recommendations.
> >
> > Its interesting comparing Wink and Chukwa. From many perspectives Chukwa
> is
> > much more active than Wink but we're about to graduate Wink and talking
> > about retiring this one. I've not yet had a chance to go through all the
> > Chukwa archives but unless i'm misunderstanding something Chukwa isn't
> just
> > a lone coder, there have been several committers in the last months and
> > while one is doing the majority of the commits many of those are actually

On Nov 27, 2012, at 9:41 PM, Eric Yang wrote:

> <snip/>

The various comparisons are distractions.  Let's focus on Chukwa and what can be done.

> If we are going to move forward, more time in incubation is not a realistic
> option.  The only way is vote for graduation and avoid the vicious cycle of
> closing the project review.

If there's an Incubator policy change that I don't know about I'm happy to hear it and reconsider my personal opinion.  If someone wants to change Incubator policy I'm happy to discuss it.  Can you not see by my message below that I am not intransigent but am willing to discuss all manner of things?

I would focus more on the Chukwa project and not spend so much time on comparing it to other projects nor making ugly innuendoes.  Look around you.  You are surrounded by a community who wants to help.
Regards,
Alan

> On Tue, Nov 27, 2012 at 7:32 AM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
>
>>
>> On Nov 27, 2012, at 7:16 AM, ant elder wrote:
>>
>>> Unless there are compelling reason to stop, i.e continuing breaches of
>>> basic ASF polices and principles, then where possible letting a poddling
>>> continue incubation or just graduate seems better to me than making them
>> go
>>> elsewhere. Its not like a small slow problem is chewing up ASF resources,
>>> but i understand not everyone here agrees with my views on that. Wink is
>> an
>>> example of poddling in similar circumstances and there we are about to
>> have
>>> decided that graduation is better than retirement. Perhaps thats a better
>>> approach. I don't recall a graduation recommendation request from the
>>> Incubator has ever been rejected by the board so perhaps the Incubator is
>>> too conservative with graduation recommendations.
>>>
>>> Its interesting comparing Wink and Chukwa. From many perspectives Chukwa
>> is
>>> much more active than Wink but we're about to graduate Wink and talking
>>> about retiring this one. I've not yet had a chance to go through all the
>>> Chukwa archives but unless i'm misunderstanding something Chukwa isn't
>> just
>>> a lone coder, there have been several committers in the last months and
>>> while one is doing the majority of the commits many of those are actually
>>> applying patches from other people, so it looks like there are a bunch of
>>> people out there working on the project and we need to find ways of
>> better
>>> integrating them into the poddling community.
>>
>> This is an interesting line of reasoning worth pursuing, IMO.  If Chukwa
>> and Wink are actually on a par with each other we should see if it make
>> sense to apply the same reasoning about Wink to Chukwa.
>>
>> Are we implicitly having a policy change with podlings, if so, should we
>> make it explicit?
>>
>>
>> Regards,
>> Alan
>>
>>

Hi,

On Tue, Nov 27, 2012 at 10:08 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
> Apache is a non-profit organization.  If we restrict our thinking model to
> metrics of how many developers, and how many patches are committed in
> pre-defeined time limit.

Chukwa has soon spent four years incubating, with the activity level
dropping off significantly after the first year or two. We have quite
a few examples of projects where such history predicts the eventual
dissolution of the entire community. There are however also a few
cases where the community has been able to revive itself.

The question here isn't about exact metrics or some predefined level
of activity that the project should be able to reach, but rather the
trajectory you're on. Is there a reasonable expectation that Chukwa
can reverse the downward trend in activity? What are you going to do
to make that happen?

As discussed, you already tried a few things to revive the community
earlier on. What makes this time different? Why would another say six
months help if the previous ones didn't?

BR,

Jukka Zitting

Hi,

On Tue, Nov 27, 2012 at 10:29 AM, Jukka Zitting <[EMAIL PROTECTED]> wrote:
> Chukwa has soon spent four years incubating, with the activity level
> dropping off significantly after the first year or two.

Sorry, correction: Chukwa has been incubating since July 2010 after
being a part of Hadoop earlier. Got confused by the longer mailing
list and commit histories. The basic premise still stands, Chukwa was
a lot more active in 2009/10 than it is now.

BR,

Jukka Zitting

Hi,

On Mon, Nov 26, 2012 at 4:52 PM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
> Even by the PPMC's comments they obliquely acknowledge that there's not much
> activity and expressed an interested in simply keeping it around with the hopes
> that something would happen; there were no concrete ideas or plans on how to
> grow the community because, by their own admission, no one has the time to
> work much on the project.

That lack of concrete plans is a good place to start. Anyone from the
community who opposes retirement should take it up on themselves to
provide such a concrete plan for example in time for next month's
report. Just like the caster of a technical veto should come up with
an alternative implementation. :-)

As an example of how this can play out, see the way we asked Kitty to
provide such a plan [1] when some members of the community opposed the
idea of retirement. In Kitty nobody stood up to the task, so a few
months later the final decision to retire the project was a pretty
easy one to make.

Another example with a different outcome is JSPWiki that had a similar
discussion at the beginning of the year, and actually a few members of
the community did start working through all the issues and have now
produced their first Apache release and seem to be on a path towards
graduation even though the project is still far below its past
activity.

[1] http://markmail.org/message/smhl3cxvrgq5cf22

BR,

Jukka Zitting

+1 to Jukka's suggestion here. The world isn't going to end if we give them
another month, and beyond that, it will give someone besides Eric the opportunity
to help cruft the plan (hopefully 2 people besides Eric, since that would mean
3 active peeps in the community).

If that plan can't be achieved in a month, I'm +1 to retire.

Cheers,
Chris

On Nov 26, 2012, at 3:21 PM, Jukka Zitting wrote:

> Hi,
>
> On Mon, Nov 26, 2012 at 4:52 PM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
>> Even by the PPMC's comments they obliquely acknowledge that there's not much
>> activity and expressed an interested in simply keeping it around with the hopes
>> that something would happen; there were no concrete ideas or plans on how to
>> grow the community because, by their own admission, no one has the time to
>> work much on the project.
>
> That lack of concrete plans is a good place to start. Anyone from the
> community who opposes retirement should take it up on themselves to
> provide such a concrete plan for example in time for next month's
> report. Just like the caster of a technical veto should come up with
> an alternative implementation. :-)
>
> As an example of how this can play out, see the way we asked Kitty to
> provide such a plan [1] when some members of the community opposed the
> idea of retirement. In Kitty nobody stood up to the task, so a few
> months later the final decision to retire the project was a pretty
> easy one to make.
>
> Another example with a different outcome is JSPWiki that had a similar
> discussion at the beginning of the year, and actually a few members of
> the community did start working through all the issues and have now
> produced their first Apache release and seem to be on a path towards
> graduation even though the project is still far below its past
> activity.
>
> [1] http://markmail.org/message/smhl3cxvrgq5cf22
>
> BR,
>
> Jukka Zitting
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

If we decide to give the podling another chance I would prefer to give it another six months rather than just one month.  I don't think that a lot can reasonably be accomplished in one month.   I would also like to see some milestones set in those six months.  If the milestones are met or not met then the decision at the end of six months will be less contentious.

Thoughts?
Regards,
Alan

On Nov 26, 2012, at 4:39 PM, Mattmann, Chris A (388J) wrote:

> +1 to Jukka's suggestion here. The world isn't going to end if we give them
> another month, and beyond that, it will give someone besides Eric the opportunity
> to help cruft the plan (hopefully 2 people besides Eric, since that would mean
> 3 active peeps in the community).
>
> If that plan can't be achieved in a month, I'm +1 to retire.
>
> Cheers,
> Chris
>
> On Nov 26, 2012, at 3:21 PM, Jukka Zitting wrote:
>
>> Hi,
>>
>> On Mon, Nov 26, 2012 at 4:52 PM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
>>> Even by the PPMC's comments they obliquely acknowledge that there's not much
>>> activity and expressed an interested in simply keeping it around with the hopes
>>> that something would happen; there were no concrete ideas or plans on how to
>>> grow the community because, by their own admission, no one has the time to
>>> work much on the project.
>>
>> That lack of concrete plans is a good place to start. Anyone from the
>> community who opposes retirement should take it up on themselves to
>> provide such a concrete plan for example in time for next month's
>> report. Just like the caster of a technical veto should come up with
>> an alternative implementation. :-)
>>
>> As an example of how this can play out, see the way we asked Kitty to
>> provide such a plan [1] when some members of the community opposed the
>> idea of retirement. In Kitty nobody stood up to the task, so a few
>> months later the final decision to retire the project was a pretty
>> easy one to make.
>>
>> Another example with a different outcome is JSPWiki that had a similar
>> discussion at the beginning of the year, and actually a few members of
>> the community did start working through all the issues and have now
>> produced their first Apache release and seem to be on a path towards
>> graduation even though the project is still far below its past
>> activity.
>>
>> [1] http://markmail.org/message/smhl3cxvrgq5cf22
>>
>> BR,
>>
>> Jukka Zitting
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

Great to hear, one month seemed too short to accomplish so much. I'd be
happy to volunteer as another mentor if some fresh eyes will help.

   ...ant

On Tue, Nov 27, 2012 at 3:32 AM, Alan Cabrera <[EMAIL PROTECTED]> wrote:

> If we decide to give the podling another chance I would prefer to give it
> another six months rather than just one month.  I don't think that a lot
> can reasonably be accomplished in one month.   I would also like to see
> some milestones set in those six months.  If the milestones are met or not
> met then the decision at the end of six months will be less contentious.
>
> Thoughts?
>
>
> Regards,
> Alan
>
> On Nov 26, 2012, at 4:39 PM, Mattmann, Chris A (388J) wrote:
>
> > +1 to Jukka's suggestion here. The world isn't going to end if we give
> them
> > another month, and beyond that, it will give someone besides Eric the
> opportunity
> > to help cruft the plan (hopefully 2 people besides Eric, since that
> would mean
> > 3 active peeps in the community).
> >
> > If that plan can't be achieved in a month, I'm +1 to retire.
> >
> > Cheers,
> > Chris
> >
> > On Nov 26, 2012, at 3:21 PM, Jukka Zitting wrote:
> >
> >> Hi,
> >>
> >> On Mon, Nov 26, 2012 at 4:52 PM, Alan Cabrera <[EMAIL PROTECTED]>
> wrote:
> >>> Even by the PPMC's comments they obliquely acknowledge that there's
> not much
> >>> activity and expressed an interested in simply keeping it around with
> the hopes
> >>> that something would happen; there were no concrete ideas or plans on
> how to
> >>> grow the community because, by their own admission, no one has the
> time to
> >>> work much on the project.
> >>
> >> That lack of concrete plans is a good place to start. Anyone from the
> >> community who opposes retirement should take it up on themselves to
> >> provide such a concrete plan for example in time for next month's
> >> report. Just like the caster of a technical veto should come up with
> >> an alternative implementation. :-)
> >>
> >> As an example of how this can play out, see the way we asked Kitty to
> >> provide such a plan [1] when some members of the community opposed the
> >> idea of retirement. In Kitty nobody stood up to the task, so a few
> >> months later the final decision to retire the project was a pretty
> >> easy one to make.
> >>
> >> Another example with a different outcome is JSPWiki that had a similar
> >> discussion at the beginning of the year, and actually a few members of
> >> the community did start working through all the issues and have now
> >> produced their first Apache release and seem to be on a path towards
> >> graduation even though the project is still far below its past
> >> activity.
> >>
> >> [1] http://markmail.org/message/smhl3cxvrgq5cf22
> >>
> >> BR,
> >>
> >> Jukka Zitting
> >>
> >> ---------------------------------------------------------------------
> >> To unsubscribe, e-mail: [EMAIL PROTECTED]
> >> For additional commands, e-mail: [EMAIL PROTECTED]
> >>
> >
> >
> > ---------------------------------------------------------------------
> > To unsubscribe, e-mail: [EMAIL PROTECTED]
> > For additional commands, e-mail: [EMAIL PROTECTED]
> >
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>

The more the merrier!  :)
Regards,
Alan

On Nov 26, 2012, at 11:50 PM, ant elder wrote:

> Great to hear, one month seemed too short to accomplish so much. I'd be
> happy to volunteer as another mentor if some fresh eyes will help.
>
>   ...ant
>
> On Tue, Nov 27, 2012 at 3:32 AM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
>
>> If we decide to give the podling another chance I would prefer to give it
>> another six months rather than just one month.  I don't think that a lot
>> can reasonably be accomplished in one month.   I would also like to see
>> some milestones set in those six months.  If the milestones are met or not
>> met then the decision at the end of six months will be less contentious.
>>
>> Thoughts?
>>
>>
>> Regards,
>> Alan
>>
>> On Nov 26, 2012, at 4:39 PM, Mattmann, Chris A (388J) wrote:
>>
>>> +1 to Jukka's suggestion here. The world isn't going to end if we give
>> them
>>> another month, and beyond that, it will give someone besides Eric the
>> opportunity
>>> to help cruft the plan (hopefully 2 people besides Eric, since that
>> would mean
>>> 3 active peeps in the community).
>>>
>>> If that plan can't be achieved in a month, I'm +1 to retire.
>>>
>>> Cheers,
>>> Chris
>>>
>>> On Nov 26, 2012, at 3:21 PM, Jukka Zitting wrote:
>>>
>>>> Hi,
>>>>
>>>> On Mon, Nov 26, 2012 at 4:52 PM, Alan Cabrera <[EMAIL PROTECTED]>
>> wrote:
>>>>> Even by the PPMC's comments they obliquely acknowledge that there's
>> not much
>>>>> activity and expressed an interested in simply keeping it around with
>> the hopes
>>>>> that something would happen; there were no concrete ideas or plans on
>> how to
>>>>> grow the community because, by their own admission, no one has the
>> time to
>>>>> work much on the project.
>>>>
>>>> That lack of concrete plans is a good place to start. Anyone from the
>>>> community who opposes retirement should take it up on themselves to
>>>> provide such a concrete plan for example in time for next month's
>>>> report. Just like the caster of a technical veto should come up with
>>>> an alternative implementation. :-)
>>>>
>>>> As an example of how this can play out, see the way we asked Kitty to
>>>> provide such a plan [1] when some members of the community opposed the
>>>> idea of retirement. In Kitty nobody stood up to the task, so a few
>>>> months later the final decision to retire the project was a pretty
>>>> easy one to make.
>>>>
>>>> Another example with a different outcome is JSPWiki that had a similar
>>>> discussion at the beginning of the year, and actually a few members of
>>>> the community did start working through all the issues and have now
>>>> produced their first Apache release and seem to be on a path towards
>>>> graduation even though the project is still far below its past
>>>> activity.
>>>>
>>>> [1] http://markmail.org/message/smhl3cxvrgq5cf22
>>>>
>>>> BR,
>>>>
>>>> Jukka Zitting
>>>>
>>>> ---------------------------------------------------------------------
>>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>>
>>>
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>>

Alan, +1 from me.

Cheers,
Chris

On Nov 26, 2012, at 10:32 PM, Alan Cabrera wrote:

> If we decide to give the podling another chance I would prefer to give it another six months rather than just one month.  I don't think that a lot can reasonably be accomplished in one month.   I would also like to see some milestones set in those six months.  If the milestones are met or not met then the decision at the end of six months will be less contentious.
>
> Thoughts?
>
>
> Regards,
> Alan
>
> On Nov 26, 2012, at 4:39 PM, Mattmann, Chris A (388J) wrote:
>
>> +1 to Jukka's suggestion here. The world isn't going to end if we give them
>> another month, and beyond that, it will give someone besides Eric the opportunity
>> to help cruft the plan (hopefully 2 people besides Eric, since that would mean
>> 3 active peeps in the community).
>>
>> If that plan can't be achieved in a month, I'm +1 to retire.
>>
>> Cheers,
>> Chris
>>
>> On Nov 26, 2012, at 3:21 PM, Jukka Zitting wrote:
>>
>>> Hi,
>>>
>>> On Mon, Nov 26, 2012 at 4:52 PM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
>>>> Even by the PPMC's comments they obliquely acknowledge that there's not much
>>>> activity and expressed an interested in simply keeping it around with the hopes
>>>> that something would happen; there were no concrete ideas or plans on how to
>>>> grow the community because, by their own admission, no one has the time to
>>>> work much on the project.
>>>
>>> That lack of concrete plans is a good place to start. Anyone from the
>>> community who opposes retirement should take it up on themselves to
>>> provide such a concrete plan for example in time for next month's
>>> report. Just like the caster of a technical veto should come up with
>>> an alternative implementation. :-)
>>>
>>> As an example of how this can play out, see the way we asked Kitty to
>>> provide such a plan [1] when some members of the community opposed the
>>> idea of retirement. In Kitty nobody stood up to the task, so a few
>>> months later the final decision to retire the project was a pretty
>>> easy one to make.
>>>
>>> Another example with a different outcome is JSPWiki that had a similar
>>> discussion at the beginning of the year, and actually a few members of
>>> the community did start working through all the issues and have now
>>> produced their first Apache release and seem to be on a path towards
>>> graduation even though the project is still far below its past
>>> activity.
>>>
>>> [1] http://markmail.org/message/smhl3cxvrgq5cf22
>>>
>>> BR,
>>>
>>> Jukka Zitting
>>>
>>> ---------------------------------------------------------------------
>>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>>> For additional commands, e-mail: [EMAIL PROTECTED]
>>>
>>
>>
>> ---------------------------------------------------------------------
>> To unsubscribe, e-mail: [EMAIL PROTECTED]
>> For additional commands, e-mail: [EMAIL PROTECTED]
>>
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>

On Nov 26, 2012, at 12:21 PM, Jukka Zitting wrote:

> Hi,
>
> On Mon, Nov 26, 2012 at 4:52 PM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
>> Even by the PPMC's comments they obliquely acknowledge that there's not much
>> activity and expressed an interested in simply keeping it around with the hopes
>> that something would happen; there were no concrete ideas or plans on how to
>> grow the community because, by their own admission, no one has the time to
>> work much on the project.
>
> That lack of concrete plans is a good place to start. Anyone from the
> community who opposes retirement should take it up on themselves to
> provide such a concrete plan for example in time for next month's
> report. Just like the caster of a technical veto should come up with
> an alternative implementation. :-)
As I mentioned in an earlier email, we did have this conversation seven months ago.  We came to a consensus to give it another try.  We even added a few committers a "bit early" with the hopes that they would infuse the project with more energy.

The vote came after many discussions over the year.  The Chukwa podling, which was started back in 2010, was given its second chance.  Unless there's something glaring that was overlooked, I'm not prepared to change my mind about its retirement.
Regards,
Alan

Hi,

On Mon, Nov 26, 2012 at 10:53 PM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
> As I mentioned in an earlier email, we did have this conversation seven
> months ago.  We came to a consensus to give it another try.  We even added
> a few committers a "bit early" with the hopes that they would infuse the project
> with more energy.

That doesn't take away the fact that there are still people who are
clearly interested in continuing work on the project. Instead of
telling the community to pick up their toys and leave, I'd much rather
ask them to come up with a credible alternative. The failure of past
attempts to grow the community does not necessarily mean that future
attempts will also fail, so I'd give the community the benefit of
doubt as long as there are new ideas and people willing to try them.

If I understand correctly the problems in Chukwa are two-fold: 1) the
community isn't diverse, i.e. there are only few people involved, and
2) the community isn't active, in that even the involved people don't
have too many cycles to spend on the project.

Thus I'd raise the following questions to Eric and others who want to
keep Chukwa alive at the ASF:

a) Is it reasonable to expect existing community members to become
more active in near future? If yes, will such increased activity be
sustainable over a longer period of time?  Why? IIUC there was some
recent legal progress that might help here. What would be the best way
to measure the expected increase in activity?

b) How do you expect to get more people involved in the project? What
concrete actions will be taken to increase the chances of new
contributors showing up? Why do you believe these things will work
better than the mentioned earlier attempts at growing the community?
Good ideas of concrete actions are for example cutting new releases,
improving project documentation, presenting the project at various
venues, simplifying the project build and initial setup, and giving
more timely answers and feedback to new users and contributors (see
also my observation from October [1]). How can we best tell whether
such efforts are working?

Coming up with good answers to such questions is not necessarily easy
(and it's fine if not all of them can yet be answered), but going
through that effort should give us a good reason to continue the
incubation of Chukwa at least for a few more months until we should
start seeing some concrete and sustainable improvements in community
activity and diversity.

[1] http://markmail.org/message/tvs4yivabvrig7ia

BR,

Jukka Zitting

On Nov 26, 2012, at 4:25 PM, Jukka Zitting wrote:

> Hi,
>
> On Mon, Nov 26, 2012 at 10:53 PM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
>> As I mentioned in an earlier email, we did have this conversation seven
>> months ago.  We came to a consensus to give it another try.  We even added
>> a few committers a "bit early" with the hopes that they would infuse the project
>> with more energy.
>
> That doesn't take away the fact that there are still people who are
> clearly interested in continuing work on the project. Instead of
> telling the community to pick up their toys and leave, I'd much rather
> ask them to come up with a credible alternative. The failure of past
> attempts to grow the community does not necessarily mean that future
> attempts will also fail, so I'd give the community the benefit of
> doubt as long as there are new ideas and people willing to try them.

OT: I don't care for the "telling the community to pick up their toys and leave" characterization.  Being a mentor is a fair amount of work.  Even more work when one is carrying out what seems to be the right thing to do even when it's unpopular.  We're definitely not taking the easy route here and remarks like yours makes things unnecessarily harder.

> If I understand correctly the problems in Chukwa are two-fold: 1) the
> community isn't diverse, i.e. there are only few people involved, and
> 2) the community isn't active, in that even the involved people don't
> have too many cycles to spend on the project.

We're talking about one developer here.  Not a few people.  If others had chimed in it would probably be a different story.  With that said I do think that you have some good questions below.

> Thus I'd raise the following questions to Eric and others who want to
> keep Chukwa alive at the ASF:
>
> a) Is it reasonable to expect existing community members to becomec vcevwoi
> more active in near future? If yes, will such increased activity be
> sustainable over a longer period of time?  Why? IIUC there was some
> recent legal progress that might help here. What would be the best way
> to measure the expected increase in activity?
>
> b) How do you expect to get more people involved in the project? What
> concrete actions will be taken to increase the chances of new
> contributors showing up? Why do you believe these things will work
> better than the mentioned earlier attempts at growing the community?
> Good ideas of concrete actions are for example cutting new releases,
> improving project documentation, presenting the project at various
> venues, simplifying the project build and initial setup, and giving
> more timely answers and feedback to new users and contributors (see
> also my observation from October [1]). How can we best tell whether
> such efforts are working?
>
> Coming up with good answers to such questions is not necessarily easy
> (and it's fine if not all of them can yet be answered), but going
> through that effort should give us a good reason to continue the
> incubation of Chukwa at least for a few more months until we should
> start seeing some concrete and sustainable improvements in community
> activity and diversity.
>
> [1] http://markmail.org/message/tvs4yivabvrig7ia
I too would love to hear answers to these questions.  More importantly I'd like to also hear them from other PPMC members in addition to Eric.
Regards,
Alan

I did a search on issues updated in the past 6 month, and here is the list:

https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CHUKWA+AND+resolution+in+%28Unresolved%2C+Fixed%2C+%22Won%27t+Fix%22%29+AND+updated+%3E%3D+-180d+ORDER+BY+reporter+ASC%2C+key+DESC

The community is moving forward with development even without my
involvement.  I have only written code for 4 of 29 JIRAs.  I don't think I
am only one that is involved in this project.  There are others that are
working on this project at their own pace.  Some people may have been turn
away from this project because the assumption that this project would be
retired soon.  I am not sure how to fix this mental block that was
implanted previously by past concerns.

If Chukwa is here to stay in Apache, I would recommend to make new releases
of Chukwa to improve the community involvement.  In terms of new features,
there are some data analysis examples of collected logs could also show
developers how to use Chukwa and Hadoop related technologies together to
generate insights from collected logs.  I volunteer to make releases and
let the community contributors steer the direction of the development.  Of
course, this plan can only carry out  base on the voting result in
general@incubator.

regards,
Eric

On Mon, Nov 26, 2012 at 12:53 PM, Alan Cabrera <[EMAIL PROTECTED]> wrote:

>
> On Nov 26, 2012, at 12:21 PM, Jukka Zitting wrote:
>
> > Hi,
> >
> > On Mon, Nov 26, 2012 at 4:52 PM, Alan Cabrera <[EMAIL PROTECTED]>
> wrote:
> >> Even by the PPMC's comments they obliquely acknowledge that there's not
> much
> >> activity and expressed an interested in simply keeping it around with
> the hopes
> >> that something would happen; there were no concrete ideas or plans on
> how to
> >> grow the community because, by their own admission, no one has the time
> to
> >> work much on the project.
> >
> > That lack of concrete plans is a good place to start. Anyone from the
> > community who opposes retirement should take it up on themselves to
> > provide such a concrete plan for example in time for next month's
> > report. Just like the caster of a technical veto should come up with
> > an alternative implementation. :-)
>
>
> As I mentioned in an earlier email, we did have this conversation seven
> months ago.  We came to a consensus to give it another try.  We even added
> a few committers a "bit early" with the hopes that they would infuse the
> project with more energy.
>
> The vote came after many discussions over the year.  The Chukwa podling,
> which was started back in 2010, was given its second chance.  Unless
> there's something glaring that was overlooked, I'm not prepared to change
> my mind about its retirement.
>
>
> Regards,
> Alan
>
>

On Mon, Nov 26, 2012 at 10:56 PM, Eric Yang <[EMAIL PROTECTED]> wrote:
> I did a search on issues updated in the past 6 month, and here is the list:
>
> https://issues.apache.org/jira/secure/IssueNavigator.jspa?reset=true&jqlQuery=project+%3D+CHUKWA+AND+resolution+in+%28Unresolved%2C+Fixed%2C+%22Won%27t+Fix%22%29+AND+updated+%3E%3D+-180d+ORDER+BY+reporter+ASC%2C+key+DESC
>
> The community is moving forward with development even without my
> involvement.  I have only written code for 4 of 29 JIRAs.  I don't think I
> am only one that is involved in this project.  There are others that are
> working on this project at their own pace.  Some people may have been turn
> away from this project because the assumption that this project would be
> retired soon.  I am not sure how to fix this mental block that was
> implanted previously by past concerns.

Obviously I as a mentor failed to help people overcome this block, but
this is a hen/egg problem.
There are always many more people lurking than contributing.
Nobody contributing means no release, means no conributions. Chukwa
always had contributions, but didn't reach a critical mass.

  Bernd

Hi IPMC,

For the past two years, Chukwa has been labelled as non-active project by
mentors, and has been put on votes for retiring this project by mentor and
IPMC.
In this year's stats, Chukwa has more activities in comparison to Apache
Wink in both mailing list traffic and resolved jiras.  Yet Chukwa has been
voted to discontinue by mentors, but Wink is voted to graduate  by the same
mentor. Here are the number of mails showed up in dev list between Apache
Chukwa and Apache Wink:

Chukka

  Nov 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201211.mbox/thread>

46

Oct 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201210.mbox/thread>

14

Sep 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201209.mbox/thread>

51

Aug 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201208.mbox/thread>

64

Jul 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201207.mbox/thread>

82

Jun 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201206.mbox/thread>

15

May 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201205.mbox/thread>

24

Apr 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201204.mbox/thread>

18

Mar 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201203.mbox/thread>

71

Feb 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201202.mbox/thread>

11

Jan 2012<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201201.mbox/thread>

60
  Wink

Nov 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201211.mbox/thread>
 18

Oct 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201210.mbox/thread>
 14

Sep 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201209.mbox/thread>
 2

Aug 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201208.mbox/thread>
 69

Jul 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201207.mbox/thread>
 5

May 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201205.mbox/thread>
 26

Apr 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201204.mbox/thread>
 24

Mar 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201203.mbox/thread>
 15

Feb 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201202.mbox/thread>
 21

Jan 2012<http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/201201.mbox/thread>
 22
And Incubation status shows:

ChukwaIncubator2010-07-14865Falsegroup-1TrueTrue<http://incubator.apache.org/projects/chukwa.html>
2012-09-10760,2,413<http://people.apache.org/committers-by-project.html#chukwa>
4True <https://svn.apache.org/repos/asf/incubator/chukwa/>True<https://issues.apache.org/jira/browse/CHUKWA>
True <http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/>True<http://mail-archives.apache.org/mod_mbox/incubator-chukwa-commits/>
True <http://incubator.apache.org/chukwa/>True<http://www.apache.org/dyn/closer.cgi/incubator/chukwa/>
True <http://www.apache.org/dist/incubator/chukwa/KEYS>True<http://www.apache.org/dyn/closer.cgi/incubator/chukwa/>
WinkIncubator2009-05-271278Falsegroup-2TrueTrue<http://incubator.apache.org/projects/wink.html>
2012-08-161010,4,616<http://people.apache.org/committers-by-project.html#wink>
3True <https://svn.apache.org/repos/asf/incubator/wink>True<https://issues.apache.org/jira/browse/wink>
True <http://mail-archives.apache.org/mod_mbox/incubator-wink-dev/>True<http://mail-archives.apache.org/mod_mbox/incubator-wink-commits/>
True <http://incubator.apache.org/wink/>True<http://www.apache.org/dyn/closer.cgi/incubator/wink/>
True <http://www.apache.org/dist/incubator/wink/KEYS>True<http://www.apache.org/dyn/closer.cgi/incubator/wink/>
In both cases, the metrics shows similar activities for both projects. The
number of issues resolved or committed by Chukwa also exceeded Apache Wink
in the past 30 days.  With respect to the voting result, but it leaves me
puzzled that why should Chukwa be retired.  When there are contributors,
and there are activities for growth.

regards,
Eric

On Nov 25, 2012, at 7:33 PM, Eric Yang wrote:

> Hi IPMC,
>
> For the past two years, Chukwa has been labelled as non-active project by
> mentors, and has been put on votes for retiring this project by mentor and
> IPMC.
> In this year's stats, Chukwa has more activities in comparison to Apache
> Wink in both mailing list traffic and resolved jiras.  Yet Chukwa has been
> voted to discontinue by mentors, but Wink is voted to graduate  by the same
> mentor. Here are the number of mails showed up in dev list between Apache
> Chukwa and Apache Wink:

Since I am the mentor that started the retirement vote on the podling I will explain my perspective.

What it comes down to is actual diverse activity.  For me, the overwhelming bulk of the work for Chukwa was being done by one person.  While looking at the raw numbers the two projects seem similar, if you scrub the threads where we discuss whether or not to retire Chukwa and also look at who's doing the actual work, it seems to me that the two projects are not exactly the same.
Regards,
Alan

On Mon, Nov 26, 2012 at 6:54 AM, Alan Cabrera <[EMAIL PROTECTED]> wrote:

>
> On Nov 25, 2012, at 7:33 PM, Eric Yang wrote:
>
> > Hi IPMC,
> >
> > For the past two years, Chukwa has been labelled as non-active project by
> > mentors, and has been put on votes for retiring this project by mentor
> and
> > IPMC.
> > In this year's stats, Chukwa has more activities in comparison to Apache
> > Wink in both mailing list traffic and resolved jiras.  Yet Chukwa has
> been
> > voted to discontinue by mentors, but Wink is voted to graduate  by the
> same
> > mentor. Here are the number of mails showed up in dev list between Apache
> > Chukwa and Apache Wink:
>
> Since I am the mentor that started the retirement vote on the podling I
> will explain my perspective.
>
> What it comes down to is actual diverse activity.  For me, the
> overwhelming bulk of the work for Chukwa was being done by one person.
>  While looking at the raw numbers the two projects seem similar, if you
> scrub the threads where we discuss whether or not to retire Chukwa and also
> look at who's doing the actual work, it seems to me that the two projects
> are not exactly the same.
>
>

I went ahead and did just that.  I read a bunch of the threads and the
pattern is that one committer does (mostly small) stuff and the few others
typically just say "meh...".

It should be emphasized that retirement != project-death-sentence.  Chukwa
might even do much better as a github project where new contributors can be
groomed more aggressively than with an apache project.

I wonder which steps were taken by mentors, community and pmc to foster a
community. I want to learn something from this case. Thanks.
26.11.2012 18:55 пользователь "Alan Cabrera" <[EMAIL PROTECTED]> написал:

>
> On Nov 25, 2012, at 7:33 PM, Eric Yang wrote:
>
> > Hi IPMC,
> >
> > For the past two years, Chukwa has been labelled as non-active project by
> > mentors, and has been put on votes for retiring this project by mentor
> and
> > IPMC.
> > In this year's stats, Chukwa has more activities in comparison to Apache
> > Wink in both mailing list traffic and resolved jiras.  Yet Chukwa has
> been
> > voted to discontinue by mentors, but Wink is voted to graduate  by the
> same
> > mentor. Here are the number of mails showed up in dev list between Apache
> > Chukwa and Apache Wink:
>
> Since I am the mentor that started the retirement vote on the podling I
> will explain my perspective.
>
> What it comes down to is actual diverse activity.  For me, the
> overwhelming bulk of the work for Chukwa was being done by one person.
>  While looking at the raw numbers the two projects seem similar, if you
> scrub the threads where we discuss whether or not to retire Chukwa and also
> look at who's doing the actual work, it seems to me that the two projects
> are not exactly the same.
>
>
> Regards,
> Alan
>
>
> ---------------------------------------------------------------------
> To unsubscribe, e-mail: [EMAIL PROTECTED]
> For additional commands, e-mail: [EMAIL PROTECTED]
>
>

On Nov 26, 2012, at 7:51 AM, Alexei Fedotov wrote:

> I wonder which steps were taken by mentors, community and pmc to foster a
> community. I want to learn something from this case. Thanks.

When we had our first retirement discussion early this year it was the consensus that we would wait until the end of the year to see if anyone would show up.  The reasons for waiting were not entirely clear to me but there was something about how the project was split off from Hadoop back in 2010.

Anyway, Eric was enthusiastic at the time and so I personally figured, sure, let's give it some more time and see what happens.

We also added a few more committers with the hopes of infusing new blood into the project.

So, we waited for more than half a year and we added new committers.  I know we don't have a dysfunctional PPMC or committership; the current set of members are great, just inactive.  Now, if not a lot of people are interested in the project itself then it's hard to whip up enthusiasm for people to contribute.

In previous podlings that I have mentored raising the specter of retirement has brought lurkers out from the background and incentivized them to step up and become active in the community.  For Chukwa no such result occurred.

So, what else can be done? We asked for plans or ideas from the PPMC and, as I mentioned in voting thread, the discussion with the mentors involved only one member of the PPMC.  The other PPMC members were mute with the exception of one member who stated that he'd be ok with it being put into the attic and the project moving to GitHub.

We could wait some more but we already did that and I don't think that it will be productive given that the project is basically flatlined.
Regards,
Alan

On Mon, Nov 26, 2012 at 7:11 PM, Alan Cabrera <[EMAIL PROTECTED]> wrote:
>
> On Nov 26, 2012, at 7:51 AM, Alexei Fedotov wrote:
>
>> I wonder which steps were taken by mentors, community and pmc to foster a
>> community. I want to learn something from this case. Thanks.
>
> When we had our first retirement discussion early this year it was the consensus that we would wait until the end of the year to see if anyone would show up.  The reasons for waiting were not entirely clear to me but there was something about how the project was split off from Hadoop back in 2010.
>
> Anyway, Eric was enthusiastic at the time and so I personally figured, sure, let's give it some more time and see what happens.
>
> We also added a few more committers with the hopes of infusing new blood into the project.
>
> So, we waited for more than half a year and we added new committers.  I know we don't have a dysfunctional PPMC or committership; the current set of members are great, just inactive.  Now, if not a lot of people are interested in the project itself then it's hard to whip up enthusiasm for people to contribute.
>
> In previous podlings that I have mentored raising the specter of retirement has brought lurkers out from the background and incentivized them to step up and become active in the community.  For Chukwa no such result occurred.
>
> So, what else can be done? We asked for plans or ideas from the PPMC and, as I mentioned in voting thread, the discussion with the mentors involved only one member of the PPMC.  The other PPMC members were mute with the exception of one member who stated that he'd be ok with it being put into the attic and the project moving to GitHub.
>
> We could wait some more but we already did that and I don't think that it will be productive given that the project is basically flatlined.

Chukwa has had a hard time and still has impediments which are
unlikely to change anytime soon.

Chukwa is based on HBase, which for a long time had no proper release,
same with Hadoop. A lot of energy was lost.
In the meantime, Flume came along and implemented a similar product.
Now, we are seeing that main contributors have issues getting their
patches through for non-Apache-related legal reasons.

Some people don't give up hope even yet, which is quite honorable. I,
however, don't expect that the situation improves in short or even
medium term.

  Bernd

On Mon, Nov 26, 2012 at 4:33 AM, Eric Yang <[EMAIL PROTECTED]> wrote:
> ...With respect to the voting result, but it leaves me
> puzzled that why should Chukwa be retired.  When there are contributors,
> and there are activities for growth...

Note that there's several -1s in the [VOTE] thread on
general@incubator.a.o, based on the lack of consensus that we see in
[1].

-Bertrand

[1] http://mail-archives.apache.org/mod_mbox/incubator-chukwa-dev/201211.mbox/%[EMAIL PROTECTED]%3E