Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> [Discussion] Any thoughts on PIG-3457?


+
Cheolsoo Park 2013-09-30, 16:51
+
Daniel Dai 2013-09-30, 17:33
+
Cheolsoo Park 2013-09-30, 17:43
+
Jeremy Karn 2013-09-30, 18:12
+
Cheolsoo Park 2013-09-30, 18:45
Copy link to this message
-
Re: [Discussion] Any thoughts on PIG-3457?
I don't mind trying to put together a patch for what I described above if
there's a general consensus on the strategy we should take (or at least no
big objections).

I think the multiple jars solution could be troublesome, but maybe after
seeing what a patch looks like for a single jar solution it'll seem nicer.
On Mon, Sep 30, 2013 at 2:45 PM, Cheolsoo Park <[EMAIL PROTECTED]> wrote:

> Hi Jeremy,
>
> What you're saying makes sense, and patch is welcome. ;-) But complexity
> comes from that there are many classes that are associated with one
> another, and it seems necessary to bring back all of them together in order
> to provide full backward compatibility.
>
> After spending many hours on the weekend, I concluded that adding more
> workarounds (classes, methods, packages, etc) to the current code makes it
> only less maintainable and readable. So I prefer a simpler approach.
>
> For eg, we can just publish two jars - pig.jar w/ old API and pig-new.jar
> w/ new API - maybe not in 0.12 but in 0.13. Since we already have a
> tez-branch, we can use it to manage the new version of classes. Then, users
> can switch to pig-new.jar gradually in 0.13 and 0.14. When we finally merge
> tez-branch into trunk, we can publish a single jar again.
>
> Of course, this is not trivial either because we have to maintain two
> branches. But I feel that managing two branches independently is easier
> than maintaining all sorts of workarounds for backward compatibility in the
> source code. In addition, we will have more flexibility in terms of
> designing new API because we will be completely free from backward
> compatibility. No?
>
> Thanks,
> Cheolsoo
>
> On Mon, Sep 30, 2013 at 11:12 AM, Jeremy Karn <[EMAIL PROTECTED]>
> wrote:
>
> > What about the option of leaving all of the MR specific logic in the
> > original classes but marking those methods as deprecated and telling
> people
> > to switch to using a MR specific object that extends the original class.
> >  So for example:
> >
> > JobStats - Reverted to being as it was before PIG-3419 but with all MR
> > specific logic deprecated.
> > MRJobStats - Would just extend JobStats.
> >
> > If we did this, external software could switch their code from using
> > JobStats to MRJobStats at their own pace and without breaking against any
> > specific version of Pig.  After a few versions the MR specific logic
> could
> > be removed from JobStats and pushed into MRJobStats and it shouldn't
> break
> > anything for people that had made that change.
> >
> > I'm not familiar with all of the changes in PIG-3419 so this might not
> work
> > everywhere.
> >
> >
> > On Mon, Sep 30, 2013 at 1:43 PM, Cheolsoo Park <[EMAIL PROTECTED]>
> > wrote:
> >
> > > To be specific, we will need to revert all the following commits in
> > order:
> > >
> > >
> > > commit ad1b87d4ba073680ad0a7fc8c76baeb8b611c982
> > > Author: Cheolsoo Park <[EMAIL PROTECTED]>
> > > Date:   Fri Sep 20 22:47:29 2013 +0000
> > >
> > >     PIG-3471: Add a base abstract class for ExecutionEngine (cheolsoo)
> > >
> > >     git-svn-id:
> > >
> > >
> >
> https://svn.apache.org/repos/asf/pig/trunk@152516513f79535-47bb-0310-9956-ffa450edef68
> > >
> > > commit 4305a6f4737d07396ae13fd95d7c1da7933b38a1
> > > Author: Jianyong Dai <[EMAIL PROTECTED]>
> > > Date:   Wed Sep 18 19:09:49 2013 +0000
> > >
> > >     PIG-3457: Provide backward compatibility for PigStatsUtil and
> > JobStats
> > >
> > >     git-svn-id:
> > >
> > >
> >
> https://svn.apache.org/repos/asf/pig/trunk@152453213f79535-47bb-0310-9956-ffa450edef68
> > >
> > > commit e85cf34c92713aa697a1cda7a9c2b3db139350f7
> > > Author: Cheolsoo Park <[EMAIL PROTECTED]>
> > > Date:   Wed Sep 18 15:37:58 2013 +0000
> > >
> > >     PIG-3464: Mark ExecType and ExecutionEngine interfaces as evolving
> > > (cheolsoo)
> > >
> > > commit fd8b7cdf9292b305f02386d560c25298ab492a0b
> > > Author: Cheolsoo Park <[EMAIL PROTECTED]>
> > > Date:   Fri Aug 30 20:04:29 2013 +0000
> > >
> > >     PIG-3419: Pluggable Execution Engine (achalsoni81 via cheolsoo)
Jeremy Karn / Lead Developer
MORTAR DATA / 519 277 4391 / www.mortardata.com
+
Daniel Dai 2013-09-30, 19:48
+
Jeremy Karn 2013-09-30, 19:56
+
Pradeep Gollakota 2013-09-30, 20:14
+
Alan Gates 2013-09-30, 20:28
+
Rohini Palaniswamy 2013-09-30, 21:26
+
Cheolsoo Park 2013-09-30, 21:42
+
Aniket Mokashi 2013-09-30, 22:37
+
Cheolsoo Park 2013-09-30, 22:52
+
Cheolsoo Park 2013-10-01, 04:28
+
Daniel Dai 2013-10-01, 05:17