Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # dev >> [Discussion] Any thoughts on PIG-3457?


Copy link to this message
-
Re: [Discussion] Any thoughts on PIG-3457?
Ok, sounds good.  I'll take a shot at it tonight.
On Mon, Sep 30, 2013 at 3:48 PM, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Thanks Jeremy. That sounds absolutely fine. The only reservation is I don't
> want to delay 0.12.0 release. We need to either do it quickly, or rollback
> PIG-3419 and then do it on 0.13.
>
> Thanks,
> Daniel
>
>
> On Mon, Sep 30, 2013 at 12:40 PM, Jeremy Karn <[EMAIL PROTECTED]>
> wrote:
>
> > I don't mind trying to put together a patch for what I described above if
> > there's a general consensus on the strategy we should take (or at least
> no
> > big objections).
> >
> > I think the multiple jars solution could be troublesome, but maybe after
> > seeing what a patch looks like for a single jar solution it'll seem
> nicer.
> >
> >
> > On Mon, Sep 30, 2013 at 2:45 PM, Cheolsoo Park <[EMAIL PROTECTED]>
> > wrote:
> >
> > > Hi Jeremy,
> > >
> > > What you're saying makes sense, and patch is welcome. ;-) But
> complexity
> > > comes from that there are many classes that are associated with one
> > > another, and it seems necessary to bring back all of them together in
> > order
> > > to provide full backward compatibility.
> > >
> > > After spending many hours on the weekend, I concluded that adding more
> > > workarounds (classes, methods, packages, etc) to the current code makes
> > it
> > > only less maintainable and readable. So I prefer a simpler approach.
> > >
> > > For eg, we can just publish two jars - pig.jar w/ old API and
> pig-new.jar
> > > w/ new API - maybe not in 0.12 but in 0.13. Since we already have a
> > > tez-branch, we can use it to manage the new version of classes. Then,
> > users
> > > can switch to pig-new.jar gradually in 0.13 and 0.14. When we finally
> > merge
> > > tez-branch into trunk, we can publish a single jar again.
> > >
> > > Of course, this is not trivial either because we have to maintain two
> > > branches. But I feel that managing two branches independently is easier
> > > than maintaining all sorts of workarounds for backward compatibility in
> > the
> > > source code. In addition, we will have more flexibility in terms of
> > > designing new API because we will be completely free from backward
> > > compatibility. No?
> > >
> > > Thanks,
> > > Cheolsoo
> > >
> > > On Mon, Sep 30, 2013 at 11:12 AM, Jeremy Karn <[EMAIL PROTECTED]>
> > > wrote:
> > >
> > > > What about the option of leaving all of the MR specific logic in the
> > > > original classes but marking those methods as deprecated and telling
> > > people
> > > > to switch to using a MR specific object that extends the original
> > class.
> > > >  So for example:
> > > >
> > > > JobStats - Reverted to being as it was before PIG-3419 but with all
> MR
> > > > specific logic deprecated.
> > > > MRJobStats - Would just extend JobStats.
> > > >
> > > > If we did this, external software could switch their code from using
> > > > JobStats to MRJobStats at their own pace and without breaking against
> > any
> > > > specific version of Pig.  After a few versions the MR specific logic
> > > could
> > > > be removed from JobStats and pushed into MRJobStats and it shouldn't
> > > break
> > > > anything for people that had made that change.
> > > >
> > > > I'm not familiar with all of the changes in PIG-3419 so this might
> not
> > > work
> > > > everywhere.
> > > >
> > > >
> > > > On Mon, Sep 30, 2013 at 1:43 PM, Cheolsoo Park <[EMAIL PROTECTED]
> >
> > > > wrote:
> > > >
> > > > > To be specific, we will need to revert all the following commits in
> > > > order:
> > > > >
> > > > >
> > > > > commit ad1b87d4ba073680ad0a7fc8c76baeb8b611c982
> > > > > Author: Cheolsoo Park <[EMAIL PROTECTED]>
> > > > > Date:   Fri Sep 20 22:47:29 2013 +0000
> > > > >
> > > > >     PIG-3471: Add a base abstract class for ExecutionEngine
> > (cheolsoo)
> > > > >
> > > > >     git-svn-id:
> > > > >
> > > > >
> > > >
> > >
> >
> https://svn.apache.org/repos/asf/pig/trunk@152516513f79535-47bb-0310-9956-ffa450edef68
> > >

Jeremy Karn / Lead Developer
MORTAR DATA / 519 277 4391 / www.mortardata.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB