Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> [Discussion] Any thoughts on PIG-3457?


+
Cheolsoo Park 2013-09-30, 16:51
Copy link to this message
-
Re: [Discussion] Any thoughts on PIG-3457?
Thanks Cheolsoo! My opinion is provide backward compatibility for PigStats
is a must, otherwise it could be a havoc. I imagine PigStats is widely used
by Pig users via PigRunner and PPNL interface. People use PigStats to
collect MR job details of the Pig job. Though PigStats is marked for
Evolving, this is mostly for extending PigStats, not limiting PigStats as
PIG-3419 did. Even if we really need to change, we need to very well
communicate with users over time, Pig 0.12 is not an option.

PIG-3457 is trying to provide a backward compatibility way for PigStats,
but just like Cheolsoo said, it is far from ideal. I now tend to agree
Rohini's suggestion on PIG-3419, rollback PIG-3419, until we find a better
way. Seems PIG-3419 is a little premature. Besides the above mentioned
PigStats issue, I've already found 2 additional issues:
1. "explain" shows unoptimized logical plan instead of optimized one
2. HangingJobKiller is removed

How does others think?

Thanks,
Daniel
On Mon, Sep 30, 2013 at 9:51 AM, Cheolsoo Park <[EMAIL PROTECTED]> wrote:

> Hi devs,
>
> PIG-3419 <https://issues.apache.org/jira/browse/PIG-3419> broke backward
> compatibility for downstream applications such as Oozie, and
> PIG-3457<https://issues.apache.org/jira/browse/PIG-3457> is
> trying to fix it.
>
> In summary, we need to keep the old MR-specific JobStats and PigStats for
> backward compatibility sadly because they're currently exposed in several
> user-facing API including PigRunner, PigServer, etc. However, this defeats
> the purpose of PIG-3419
> <https://issues.apache.org/jira/browse/PIG-3419> because
> we cannot implement non-MR execution engines in the back-end if the
> front-end API is tied to MR.
>
> Daniel and I were having a discussion in the jira, but it seems more
> complicated than I thought. I am wondering whether anyone has a good
> suggestion on how to solve it. This is blocking the 0.12 release.
>
> Thanks,
> Cheolsoo
>

--
CONFIDENTIALITY NOTICE
NOTICE: This message is intended for the use of the individual or entity to
which it is addressed and may contain information that is confidential,
privileged and exempt from disclosure under applicable law. If the reader
of this message is not the intended recipient, you are hereby notified that
any printing, copying, dissemination, distribution, disclosure or
forwarding of this communication is strictly prohibited. If you have
received this communication in error, please contact the sender immediately
and delete it from your system. Thank You.
+
Cheolsoo Park 2013-09-30, 17:43
+
Jeremy Karn 2013-09-30, 18:12
+
Cheolsoo Park 2013-09-30, 18:45
+
Jeremy Karn 2013-09-30, 19:40
+
Daniel Dai 2013-09-30, 19:48
+
Jeremy Karn 2013-09-30, 19:56
+
Pradeep Gollakota 2013-09-30, 20:14
+
Alan Gates 2013-09-30, 20:28
+
Rohini Palaniswamy 2013-09-30, 21:26
+
Cheolsoo Park 2013-09-30, 21:42
+
Aniket Mokashi 2013-09-30, 22:37
+
Cheolsoo Park 2013-09-30, 22:52
+
Cheolsoo Park 2013-10-01, 04:28
+
Daniel Dai 2013-10-01, 05:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB