Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> [Discussion] Keep or revert PIG-3419 in trunk?


+
Cheolsoo Park 2013-10-07, 19:05
+
Daniel Dai 2013-10-07, 21:49
Copy link to this message
-
Re: [Discussion] Keep or revert PIG-3419 in trunk?
I'm +1 on reverting PIG-3419 on trunk and then trying to redo it in a
better way.  Either way it would be good to get this resolved soon.
On Mon, Oct 7, 2013 at 5:49 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Let me first revert PIG-3457, that complicates things. I will try to
> explore for a solution in next the few days. But if we cannot settle
> quickly, I would suggest to revert PIG-3419 on trunk first, and redo it
> later in a better way. Otherwise, it will be very hard to revert.
>
> Thanks,
> Daniel
>
>
> On Mon, Oct 7, 2013 at 12:05 PM, Cheolsoo Park <[EMAIL PROTECTED]>
> wrote:
>
> > Hi devs,
> >
> > This is a follow-up discussion about how to resolve the backward
> > incompatibility of PIG-3419 (Pluggable execution engine). Per the
> previous
> > discussion <http://search-hadoop.com/m/wYz6hz9CoE>, I reverted it in
> 0.12
> > but kept it in trunk. As we keep committing more changes into trunk, it
> > gets harder to back out PIG-3419 cleanly. So I suggest we should make a
> > decision sooner rather than later.
> >
> > The crux of the problem is as follows:
> >
> > PIG-3419 removes all the MR-specific things from JobStats. However,
> > PigRunner and PigServer returns PigStats that in turn exposes JobStats to
> > end users. So changing JobStats breaks backward compatibility for
> > downstream projects such as Oozie. While changing the semantics of
> JobStats
> > is acceptable, we must provide a deprecation path so that end users can
> > upgrade their applications smoothly.
> >
> > Proposed solutions:
> >
> > 1) Provide backward compatibility in source code: Maybe possible, but no
> > one has come up with a clean solution. For eg, I failed. :(
> >
> > 2) Publish two jars: We keep PIG-3419 only in tez-branch and publish two
> > jars (one for old API and one for new API) for a couple of future
> releases.
> > If we do this, we're going to revert PIG-3419 in trunk.
> >
> > What do you think?
> >
> > Thanks,
> > Cheolsoo
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>

--

Jeremy Karn / Lead Developer
MORTAR DATA / 519 277 4391 / www.mortardata.com
+
Daniel Dai 2013-10-15, 20:54
+
Cheolsoo Park 2013-10-17, 17:34