Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig, mail # dev - [Discussion] Keep or revert PIG-3419 in trunk?


+
Cheolsoo Park 2013-10-07, 19:05
Copy link to this message
-
Re: [Discussion] Keep or revert PIG-3419 in trunk?
Daniel Dai 2013-10-07, 21:49
Let me first revert PIG-3457, that complicates things. I will try to
explore for a solution in next the few days. But if we cannot settle
quickly, I would suggest to revert PIG-3419 on trunk first, and redo it
later in a better way. Otherwise, it will be very hard to revert.

Thanks,
Daniel
On Mon, Oct 7, 2013 at 12:05 PM, Cheolsoo Park <[EMAIL PROTECTED]> wrote:

> Hi devs,
>
> This is a follow-up discussion about how to resolve the backward
> incompatibility of PIG-3419 (Pluggable execution engine). Per the previous
> discussion <http://search-hadoop.com/m/wYz6hz9CoE>, I reverted it in 0.12
> but kept it in trunk. As we keep committing more changes into trunk, it
> gets harder to back out PIG-3419 cleanly. So I suggest we should make a
> decision sooner rather than later.
>
> The crux of the problem is as follows:
>
> PIG-3419 removes all the MR-specific things from JobStats. However,
> PigRunner and PigServer returns PigStats that in turn exposes JobStats to
> end users. So changing JobStats breaks backward compatibility for
> downstream projects such as Oozie. While changing the semantics of JobStats
> is acceptable, we must provide a deprecation path so that end users can
> upgrade their applications smoothly.
>
> Proposed solutions:
>
> 1) Provide backward compatibility in source code: Maybe possible, but no
> one has come up with a clean solution. For eg, I failed. :(
>
> 2) Publish two jars: We keep PIG-3419 only in tez-branch and publish two
> jars (one for old API and one for new API) for a couple of future releases.
> If we do this, we're going to revert PIG-3419 in trunk.
>
> What do you think?
>
> Thanks,
> Cheolsoo
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
+
Jeremy Karn 2013-10-15, 19:40
+
Daniel Dai 2013-10-15, 20:54
+
Cheolsoo Park 2013-10-17, 17:34