Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> [Discussion] Any thoughts on PIG-3457?


Copy link to this message
-
Re: [Discussion] Any thoughts on PIG-3457?
Thanks Aniket.

I'll revert the aforementioned commits in 0.12 tonight. I will leave them
in trunk until we decide what to do.
On Mon, Sep 30, 2013 at 3:37 PM, Aniket Mokashi <[EMAIL PROTECTED]> wrote:

> +1 on reverting PIG-3419 and applying it to tez branch if its blocking
> pig-0.12 release.
>
> On Mon, Sep 30, 2013 at 2:42 PM, Cheolsoo Park <[EMAIL PROTECTED]>
> wrote:
>
> > I am waiting for +1 from Twitter.
> >
> > Like Alan suggested, let's revert PIG-3419 et al in 0.12 first. Then, we
> > can decide what to do in trunk. I volunteer to do grunt work since I am
> the
> > one who committed them.
> >
> >
> > On Mon, Sep 30, 2013 at 2:26 PM, Rohini Palaniswamy <
> > [EMAIL PROTECTED]
> > > wrote:
> >
> > > +1. I was already asking for keeping the new API changes only in Tez
> > branch
> > > till it evolves and is finalized, so I have no objections to reverting
> > it.
> > >
> > > Regards,
> > > Rohini
> > >
> > >
> > > On Mon, Sep 30, 2013 at 1:28 PM, Alan Gates <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > We should separate out two separate concerns.  If I understand
> > correctly
> > > > we don't need any of these changes in 0.12.  So we should revert
> these
> > > > patches from the 12 branch so that we can get it released quickly in
> a
> > > > backwards compatible way.
> > > >
> > > > We will then have plenty of time to discuss the separate question of
> > how
> > > > we proceed going forward (deprecated APIs or new APIs).
> > > >
> > > > Alan.
> > > >
> > > > On Sep 30, 2013, at 11:45 AM, Cheolsoo Park wrote:
> > > >
> > > > > Hi Jeremy,
> > > > >
> > > > > What you're saying makes sense, and patch is welcome. ;-) But
> > > complexity
> > > > > comes from that there are many classes that are associated with one
> > > > > another, and it seems necessary to bring back all of them together
> in
> > > > order
> > > > > to provide full backward compatibility.
> > > > >
> > > > > After spending many hours on the weekend, I concluded that adding
> > more
> > > > > workarounds (classes, methods, packages, etc) to the current code
> > makes
> > > > it
> > > > > only less maintainable and readable. So I prefer a simpler
> approach.
> > > > >
> > > > > For eg, we can just publish two jars - pig.jar w/ old API and
> > > pig-new.jar
> > > > > w/ new API - maybe not in 0.12 but in 0.13. Since we already have a
> > > > > tez-branch, we can use it to manage the new version of classes.
> Then,
> > > > users
> > > > > can switch to pig-new.jar gradually in 0.13 and 0.14. When we
> finally
> > > > merge
> > > > > tez-branch into trunk, we can publish a single jar again.
> > > > >
> > > > > Of course, this is not trivial either because we have to maintain
> two
> > > > > branches. But I feel that managing two branches independently is
> > easier
> > > > > than maintaining all sorts of workarounds for backward
> compatibility
> > in
> > > > the
> > > > > source code. In addition, we will have more flexibility in terms of
> > > > > designing new API because we will be completely free from backward
> > > > > compatibility. No?
> > > > >
> > > > > Thanks,
> > > > > Cheolsoo
> > > > >
> > > > > On Mon, Sep 30, 2013 at 11:12 AM, Jeremy Karn <
> [EMAIL PROTECTED]>
> > > > wrote:
> > > > >
> > > > >> What about the option of leaving all of the MR specific logic in
> the
> > > > >> original classes but marking those methods as deprecated and
> telling
> > > > people
> > > > >> to switch to using a MR specific object that extends the original
> > > class.
> > > > >> So for example:
> > > > >>
> > > > >> JobStats - Reverted to being as it was before PIG-3419 but with
> all
> > MR
> > > > >> specific logic deprecated.
> > > > >> MRJobStats - Would just extend JobStats.
> > > > >>
> > > > >> If we did this, external software could switch their code from
> using
> > > > >> JobStats to MRJobStats at their own pace and without breaking
> > against
> > > > any
> > > > >> specific version of Pig.  After a few versions the MR specific
> logic