Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> [DISCUSSION] Pig.next

Copy link to this message

Getting to the serious points.

What are the user facing items? I have listed a few below. Please feel free to add if I have missed out on anything.

1. The language syntax
2. The language semantics
3. UDFs (EvalFunc, Load, Store, Algebraic, Accumulator, etc.)
4. Java APIs (PigServer, etc.)

In the past, we have agreed that Pig will support Hadoop APIs. I think its very important to understand when Hadoop will stabilize the APIs. It will have an impact on the APIs that we expose to our users (e.g., input and output formats).

I strongly believe that this is an important input in the decision making process, especially wrt backward compatibility.


-----Original Message-----
From: Alan Gates [mailto:[EMAIL PROTECTED]]
Sent: Thursday, March 03, 2011 10:44 AM
Subject: Re: [DISCUSSION] Pig.next

I agree that there will probably need to be several 0.9.x releases as the new optimization and parser work mature.  As a consequence of this it may be longer between 0.9 and Pig.next then there has been between the last few releases.  That only delays the question of what we call Pig.next, it does not answer it.

To me, declaring 1.0 would mean the following things:

1) Pig is ready for production use, at least by the brave.
2) It is still rough around the edges, you do not get a smooth product until 2.0 or later.
3) We will not make non-backward compatible changes to interfaces we have declared stable.

Pig is in use in production in multiple places, I do not think anyone will argue that it is not rough around the edges, and because we have users who run tens of thousands of Pig jobs daily non-backward compatible changes are impossible anyway.

As for waiting for Hadoop to go 1.0, that is like waiting for Congress to fix social security.  I am sure they will get there, but I may be retired first.  In all seriousness, the Hadoop project has not been moving with speed or agility over the last few years, and I do not think waiting for them to do something is a good idea.  Nor do I see it as necessary.  Before we could go 1.0 would we insist that every jar we import is >= 1.0?  Yes we are bound more tightly to Hadoop then we are to log4j.  But we are still our own project.  1.0 is a claim we are making about ourselves, not about the platform we run on.  We should choose our release numbering in a way that sends a clear message to our users, and let those same users evaluate Hadoop separately.

Also the argument that we should not go 1.0 because we are changing a lot of things is bogus.  We are always changing a lot of things.  If 1.0 means we will not make any major changes, then we will not get there until we go into some kinds of maintenance mode where we deem the majority of the work to have been done.  I hope I have retired before we reach that state.

My perspective on what 1.0 means obviously comes from a developer inside the project.  I would be interested in hearing from users and anyone with a more marketing oriented perspective on what message 1.0 would send to (potential) pig users.


On Mar 2, 2011, at 6:31 PM, Dmitriy Ryaboy wrote:

> I am worried that the new optimization plan work has not had a  
> chance to
> settle in, and we are releasing a brand new parser for the language  
> in 0.9.
> Those are pretty significant changes, if the idea behind calling  
> something a
> "1.0" is stability, we may want to give them a release to mature a  
> bit. Of
> course we can just release 0.9x for a while until we feel this stuff  
> has
> been tested in a wide enough variety of installations / hadoop
> configurations / use cases.
> D
> On Wed, Mar 2, 2011 at 4:52 PM, Olga Natkovich <[EMAIL PROTECTED]>  
> wrote:
>> Pig Users and Developers,
>> We are starting to plan the work after Pig 0.9. One thing we need  
>> to decide
>> is what name/number to give to the next release: Pig 0.10 or Pig 1.0.
>> I believe that we are ready to declare 1.0. Here are my reasons:
>> (1)     We are mature enough and produce good quality releases