Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # general - [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project


Copy link to this message
-
Re: [DISCUSS] Spin out MR, HDFS and YARN as their own TLPs and disband Hadoop umbrella project
Eric Baldeschwieler 2012-09-01, 00:23
Hi Folks,

Lots of good points raised here.  I remain convinced that the time to split Hadoop into TLPs in here, but I think we should also consider the practical concerns raised.

Hadoop 2.0 has been years of work in the making and is finally relatively close.  I think it would be a mistake to throw another impediment in the way of getting a stable version of 2.0 done, as many folks have pointed out.  So I'd suggest that we plan to do a split once there is a broad consensus that 2.0 is stable and widely deployed.

Perhaps folks interested in planning a split or concerned that a split might impact them can meet to refine a proposal that we can all consider implementing once 2.0 is stable.

What do folks think?

Thanks,

E14

On Aug 31, 2012, at 9:08 AM, Mattmann, Chris A (388J) wrote:

> Hey Doug,
>
> On Aug 31, 2012, at 9:00 AM, Doug Cutting wrote:
>
>> On Fri, Aug 31, 2012 at 8:09 AM, Mattmann, Chris A (388J)
>> <[EMAIL PROTECTED]> wrote:
>>> I am saying that the current members of the Apache Software Foundation's Hadoop
>>> Project Management Committee exhibit the characteristics (not just during
>>> discrete events; it's been happening for a long time) of folks who in reality
>>> shouldn't belong to the same project management committee. Note: this is
>>> NOT a bad thing. There are probably plenty of (sub-)sets of groups at Apache
>>> and elsewhere that folks wouldn't fit in to. I've enumerated some of
>>> those characteristics that you can see sometimes spill over
>>> (meta thought discussions about moving things around; or drawing arbitrary
>>> lines around pieces of code that really have nothing to do with technical
>>> stuff, and more to do about insulating and control;),
>>
>> Hadoop's community is not perfect.  But the divisions in the community
>> are not primarily aligned with subcomponent boundaries.  A project
>> split will thus not likely fix the majority of these community
>> imperfections.  It may fix some, but ought to be pursued carefully so
>> that it doesn't cause more harm than good.
>
> My own personal opinion of this is that yeah they aren't necessarily
> aligned subcomponent boundaries too so +1 agree with you.
>
>>
>>> but there are also other
>>> concerns such as frameworks put in to place (exclusivity amongst others)
>>> that themselves are pretty high indicators that this is an umbrella project.
>>
>> The partitioning of committers has now been removed in a separate
>> vote.  Hadoop is not a classic umbrella project.
>
> Despite me thinking that's a band-aid it's probably at least a good start.
> Let's hope it leads to some better interactions amongst the community
> members and to better health overall.
>
> Cheers,
> Chris
>
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Chris Mattmann, Ph.D.
> Senior Computer Scientist
> NASA Jet Propulsion Laboratory Pasadena, CA 91109 USA
> Office: 171-266B, Mailstop: 171-246
> Email: [EMAIL PROTECTED]
> WWW:   http://sunset.usc.edu/~mattmann/
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
> Adjunct Assistant Professor, Computer Science Department
> University of Southern California, Los Angeles, CA 90089 USA
> ++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++++
>