Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # dev >> [Discussion] Any thoughts on PIG-3457?


+
Cheolsoo Park 2013-09-30, 16:51
+
Daniel Dai 2013-09-30, 17:33
Copy link to this message
-
Re: [Discussion] Any thoughts on PIG-3457?
To be specific, we will need to revert all the following commits in order:
commit ad1b87d4ba073680ad0a7fc8c76baeb8b611c982
Author: Cheolsoo Park <[EMAIL PROTECTED]>
Date:   Fri Sep 20 22:47:29 2013 +0000

    PIG-3471: Add a base abstract class for ExecutionEngine (cheolsoo)

    git-svn-id:
https://svn.apache.org/repos/asf/pig/trunk@152516513f79535-47bb-0310-9956-ffa450edef68

commit 4305a6f4737d07396ae13fd95d7c1da7933b38a1
Author: Jianyong Dai <[EMAIL PROTECTED]>
Date:   Wed Sep 18 19:09:49 2013 +0000

    PIG-3457: Provide backward compatibility for PigStatsUtil and JobStats

    git-svn-id:
https://svn.apache.org/repos/asf/pig/trunk@152453213f79535-47bb-0310-9956-ffa450edef68

commit e85cf34c92713aa697a1cda7a9c2b3db139350f7
Author: Cheolsoo Park <[EMAIL PROTECTED]>
Date:   Wed Sep 18 15:37:58 2013 +0000

    PIG-3464: Mark ExecType and ExecutionEngine interfaces as evolving
(cheolsoo)

commit fd8b7cdf9292b305f02386d560c25298ab492a0b
Author: Cheolsoo Park <[EMAIL PROTECTED]>
Date:   Fri Aug 30 20:04:29 2013 +0000

    PIG-3419: Pluggable Execution Engine (achalsoni81 via cheolsoo)

    git-svn-id:
https://svn.apache.org/repos/asf/pig/trunk@151906213f79535-47bb-0310-9956-ffa450edef68
On Mon, Sep 30, 2013 at 10:33 AM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Thanks Cheolsoo! My opinion is provide backward compatibility for PigStats
> is a must, otherwise it could be a havoc. I imagine PigStats is widely used
> by Pig users via PigRunner and PPNL interface. People use PigStats to
> collect MR job details of the Pig job. Though PigStats is marked for
> Evolving, this is mostly for extending PigStats, not limiting PigStats as
> PIG-3419 did. Even if we really need to change, we need to very well
> communicate with users over time, Pig 0.12 is not an option.
>
> PIG-3457 is trying to provide a backward compatibility way for PigStats,
> but just like Cheolsoo said, it is far from ideal. I now tend to agree
> Rohini's suggestion on PIG-3419, rollback PIG-3419, until we find a better
> way. Seems PIG-3419 is a little premature. Besides the above mentioned
> PigStats issue, I've already found 2 additional issues:
> 1. "explain" shows unoptimized logical plan instead of optimized one
> 2. HangingJobKiller is removed
>
> How does others think?
>
> Thanks,
> Daniel
>
>
>
>
> On Mon, Sep 30, 2013 at 9:51 AM, Cheolsoo Park <[EMAIL PROTECTED]>
> wrote:
>
> > Hi devs,
> >
> > PIG-3419 <https://issues.apache.org/jira/browse/PIG-3419> broke backward
> > compatibility for downstream applications such as Oozie, and
> > PIG-3457<https://issues.apache.org/jira/browse/PIG-3457> is
> > trying to fix it.
> >
> > In summary, we need to keep the old MR-specific JobStats and PigStats for
> > backward compatibility sadly because they're currently exposed in several
> > user-facing API including PigRunner, PigServer, etc. However, this
> defeats
> > the purpose of PIG-3419
> > <https://issues.apache.org/jira/browse/PIG-3419> because
> > we cannot implement non-MR execution engines in the back-end if the
> > front-end API is tied to MR.
> >
> > Daniel and I were having a discussion in the jira, but it seems more
> > complicated than I thought. I am wondering whether anyone has a good
> > suggestion on how to solve it. This is blocking the 0.12 release.
> >
> > Thanks,
> > Cheolsoo
> >
>
> --
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity to
> which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
>
+
Jeremy Karn 2013-09-30, 18:12
+
Cheolsoo Park 2013-09-30, 18:45
+
Jeremy Karn 2013-09-30, 19:40
+
Daniel Dai 2013-09-30, 19:48
+
Jeremy Karn 2013-09-30, 19:56
+
Pradeep Gollakota 2013-09-30, 20:14
+
Alan Gates 2013-09-30, 20:28
+
Rohini Palaniswamy 2013-09-30, 21:26
+
Cheolsoo Park 2013-09-30, 21:42
+
Aniket Mokashi 2013-09-30, 22:37
+
Cheolsoo Park 2013-09-30, 22:52
+
Cheolsoo Park 2013-10-01, 04:28
+
Daniel Dai 2013-10-01, 05:17
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB