Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop >> mail # general >> bringing the codebases back in line


+
Ian Holsman 2010-10-21, 19:13
+
Allen Wittenauer 2010-10-21, 19:30
+
Ian Holsman 2010-10-21, 21:53
+
Allen Wittenauer 2010-10-21, 23:37
+
Konstantin Boudnik 2010-10-22, 00:10
+
Steve Loughran 2010-10-22, 11:40
+
Konstantin Boudnik 2010-10-22, 16:36
+
Steve Loughran 2010-10-22, 11:37
+
Owen OMalley 2010-10-21, 20:00
+
Ian Holsman 2010-10-21, 21:00
+
Owen OMalley 2010-10-21, 21:41
+
Eli Collins 2010-10-21, 21:54
+
Owen OMalley 2010-10-21, 22:19
+
Jakob Homan 2010-10-21, 22:30
+
Eli Collins 2010-10-22, 00:17
+
Arun C Murthy 2010-10-22, 00:31
+
Eli Collins 2010-10-22, 00:51
+
Arun C Murthy 2010-10-22, 00:57
+
Allen Wittenauer 2010-10-22, 06:46
+
Eli Collins 2010-10-30, 18:14
+
Steve Loughran 2010-10-22, 11:47
+
Doug Cutting 2010-10-21, 22:19
+
Owen OMalley 2010-10-21, 22:42
+
Ian Holsman 2010-10-21, 23:50
+
Arun C Murthy 2010-10-22, 00:30
+
Eli Collins 2010-10-22, 00:37
+
Milind A Bhandarkar 2010-10-22, 00:42
+
Ian Holsman 2010-10-22, 00:54
+
Milind A Bhandarkar 2010-10-22, 04:29
+
Konstantin Shvachko 2010-10-22, 17:36
+
Milind A Bhandarkar 2010-10-22, 17:47
Copy link to this message
-
Re: bringing the codebases back in line

On Oct 22, 2010, at 10:36 AM, Konstantin Shvachko wrote:

> Milind's point is valid, the PMC cannot demand or control what Yahoo,
> Facebook, et. al. run in their productions, or what Couldera sells  
> to their
> customers  AS  LONG  AS  it is within the Apache licensing  
> requirements.
>
> What Apache Hadoop can and should provide is a *steady* stream of base
> A-releases.
>
> I think that a single fact that we missed to release Hadoop 0.21  
> late last
> year got us into the state we are in now. As it let different Hadoop
> installations to diverge drastically from each other, whether it was  
> based
> on production or commercial reasons.
>
> Now that we have that, it would not be feasible or worthwhile to  
> find the
> common denominator based on the old 0.20 version, unless we want to  
> spend
> another year looking for it and diverging the individual  
> installations even
> more in the process.
>
> So the question imo is not "how we merge the cloudera and yahoo
> distributions", but when/how do we make the new 0.22 release.
> And how do we provide a steady release cycle after that.

+1

sanjay
>
> --Konstantin
>
> On Thu, Oct 21, 2010 at 9:29 PM, Milind A Bhandarkar
> <[EMAIL PROTECTED]>wrote:
>
>>>>
>>> right.. the trunk is not for production use.  I wasn't suggesting  
>>> that.
>>
>> So, what are you suggesting ? That Yahoo distribution of Hadoop  
>> should
>> *not* be the version we run on our production clusters ?
>>
>>>
>>> but the trunk is what will eventually become the next release.
>>
>>>
>>> Then someone in yahoo will have to decide if they are going to  
>>> move to
>>> rebase their production cluster to 0.21, or just continue back-
>>> porting
>> what
>>> they need to the version they are running on their clusters.
>>
>> Yes, that is what we do now. If there are committed patches in  
>> trunk that
>> do not scale for our needs, or break existing applications, or are  
>> deemed
>> not worth the efforts needed to backport, we do not include them in  
>> our
>> deployments, and therefore do not include in Yahoo distribution.
>>
>>>
>>> and if yahoo fixes a bug in their version, it would need to be
>>> forward-ported over to the current trunk. which will get harder and
>> harder
>>> as the paths diverge.
>>
>> Yes, indeed. So, care must be taken that paths do not diverge too  
>> much. I
>> have seen some cases where the bug fixes did not need to be forward  
>> ported,
>> because that piece of code was completely re-written in trunk.
>>
>>>
>>> I'm sure you've seen it happen on other projects when a major branch
>> lands
>>> on the trunk, and the amount of effort it takes to reconcile them.
>>
>> Yes. And that results in delayed releases. An unexpected benefit for
>> application developers was that they could spend time adding  
>> features to
>> their applications, rather than porting same applications from
>> release-to-release, and validating releases. So, it's not always bad.
>>
>> - Milind
>>
>>
>> --
>> Milind Bhandarkar
>> (mailto:[EMAIL PROTECTED])
>> (phone: 408-203-5213 W)
>>
>>
>>
+
Eli Collins 2010-10-22, 21:52
+
Konstantin Boudnik 2010-10-22, 22:03
+
Mahadev Konar 2010-10-22, 22:22
+
Ian Holsman 2010-10-23, 01:33
+
Milind A Bhandarkar 2010-10-23, 03:04
+
Bernd Fondermann 2010-10-23, 09:24
+
Milind A Bhandarkar 2010-10-23, 18:25
+
Ian Holsman 2010-10-23, 10:44
+
Milind A Bhandarkar 2010-10-23, 18:23
+
Owen OMalley 2010-10-23, 03:40
+
Bernd Fondermann 2010-10-23, 09:28
+
Allen Wittenauer 2010-10-23, 15:48
+
Bernd Fondermann 2010-10-23, 19:20
+
Owen OMalley 2010-10-23, 16:10
+
Bernd Fondermann 2010-10-23, 19:24
+
Eli Collins 2010-10-23, 19:03
+
Bernd Fondermann 2010-10-23, 08:48
+
Eli Collins 2010-10-23, 18:50
+
Steve Loughran 2010-10-22, 11:45
+
Vijay Madhavan 2010-10-23, 22:26
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB