Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Hadoop, mail # dev - Re: Heads up - 2.0.5-beta


+
Suresh Srinivas 2013-04-26, 01:36
+
Arun C Murthy 2013-04-26, 18:02
+
Konstantin Shvachko 2013-04-26, 23:34
+
Arun C Murthy 2013-04-27, 01:06
+
Konstantin Shvachko 2013-04-30, 23:28
+
Arun C Murthy 2013-05-01, 20:15
+
Konstantin Shvachko 2013-05-01, 23:08
+
Arun C Murthy 2013-05-02, 01:24
+
Chris Douglas 2013-05-02, 07:07
+
Konstantin Shvachko 2013-05-02, 09:11
+
Chris Douglas 2013-05-03, 00:53
+
Konstantin Shvachko 2013-05-02, 09:08
+
Arun C Murthy 2013-05-02, 22:05
+
Konstantin Shvachko 2013-05-03, 08:06
+
Robert Evans 2013-05-03, 15:23
+
Konstantin Shvachko 2013-05-01, 19:13
+
Arpit Agarwal 2013-04-26, 23:52
+
Amir Sanjar 2013-04-26, 02:23
+
Steve Loughran 2013-04-29, 22:37
+
Amir Sanjar 2013-04-30, 02:05
+
Steve Loughran 2013-04-30, 16:48
+
Amir Sanjar 2013-04-30, 17:09
+
Roman Shaposhnik 2013-04-26, 02:31
+
Arun C Murthy 2013-04-26, 18:15
+
Eli Collins 2013-04-26, 19:07
+
Suresh Srinivas 2013-04-26, 21:42
+
Eli Collins 2013-04-26, 22:09
Copy link to this message
-
Re: Heads up - 2.0.5-beta
Luke Lu 2013-04-26, 20:37
If protocol compatibility of v2 and v3 is a goal, HADOOP-8990 should be a
blocker for v2.

__Luke

On Fri, Apr 26, 2013 at 12:07 PM, Eli Collins <[EMAIL PROTECTED]> wrote:

> On Fri, Apr 26, 2013 at 11:15 AM, Arun C Murthy <[EMAIL PROTECTED]>
> wrote:
> >
> > On Apr 25, 2013, at 7:31 PM, Roman Shaposhnik wrote:
> >
> >> On Thu, Apr 25, 2013 at 6:34 PM, Arun C Murthy <[EMAIL PROTECTED]>
> wrote:
> >>
> >>> With that in mind, I really want to make a serious push to lock down
> APIs and wire-protocols for hadoop-2.0.5-beta.
> >>> Thus, we can confidently support hadoop-2.x in a compatible manner in
> the future. So, it's fine to add new features,
> >>> but please ensure that all APIs are frozen for hadoop-2.0.5-beta
> >>
> >> Arun, since it sounds like you have a pretty definite idea
> >> in mind for what you want 'beta' label to actually mean,
> >> could you, please, share the exact criteria?
> >
> > Sorry, I'm not sure if this is exactly what you are looking for but, as
> I mentioned above, the primary aim would be make the final set of required
> API/write-protocol changes so that we can call it a 'beta' i.e. once
> 2.0.5-beta ships users & downstream projects can be confident about forward
> compatibility in hadoop-2.x line. Obviously, we might discover a blocker
> bug post 2.0.5 which *might* necessitate an unfortunate change - but that
> should be an outstanding exception.
>
> Arun, Suresh,
>
> Mind reviewing the following page Karthik put together on
> compatibility?   http://wiki.apache.org/hadoop/Compatibility
>
> I think we should do something similar to what Sanjay proposed in
> HADOOP-5071 for Hadoop v2.   If we get on the same page on
> compatibility terms/APIs then we can quickly draft the policy, at
> least for the things we've already got consensus on.  I think our new
> developers, users, downstream projects, and partners would really
> appreciate us making this clear.  If people like the content we can
> move it to the Hadoop website and maintain it in svn like the bylaws.
>
> The reason I think we need to do so is because there's been confusion
> about what types of compatibility we promise and some open questions
> which I'm not sure everyone is clear on. Examples:
> - Are we going to preserve Hadoop v3 clients against v2 servers now
> that we have protobuf support?  (I think so..)
> - Can we break rolling upgrade of daemons in updates post GA? (I don't
> think so..)
> - Do we disallow HDFS metadata changes that require an HDFS upgrade in
> an update? (I think so..)
> - Can we remove methods from v2 and v2 updates that were deprecated in
> v0.20-22?  (Unclear)
> - Will we preserve binary compatibility for MR2 going forward? (I think
> so..)
> - Does the ability to support multiple versions of MR simultaneously
> via MR2 change the MR API compatibility story? (I don't think so..)
> - Are the RM protocols sufficiently stable to disallow incompatible
> changes potentially required by non-MR projects? (Unclear, most large
> Yarn deployments I'm aware of are running 0.23, not v2 alphas)
>
> I'm also not sure there's currently consensus on what an incompatible
> change is. For example, I think HADOOP-9151 is incompatible because it
> broke client/server wire compatibility with previous releases and any
> change that breaks wire compatibility is incompatible.  Suresh felt it
> was not an incompatible change because it did not affect API
> compatibility (ie PB is not considered part of the API) and the change
> occurred while v2 is in alpha.  Not sure we need to go through the
> whole exercise of what's allowed in an alpha and beta (water under the
> bridge, hopefully), but I do think we should clearly define an
> incompatible change.  It's fine that v2 has been a bit wild wild west
> in the alpha development stage but I think we need to get a little
> more rigorous.
>
> Thanks,
> Eli
>
+
Arun C Murthy 2013-04-27, 01:27
+
Konstantin Shvachko 2013-04-26, 23:00