For all those of you who would want to help with testing
Hadoop 0.22 release, I've setup a job on Bigtop to
produce an entire Hadoop stack on top of 0.22.

The good news is that it is now super simple to give it
all a spin. If you are on Ubuntu/Debian you can try:
   $ cd /etc/apt/sources.list.d
   $ sudo wget -O bigtop.list
http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22-ubuntu-lucid/lastSuccessfulBuild/artifact/output/bigtop.list
   $ sudo apt-get update
   $ sudo apt-get install 'hadoop*' 'hbase*'
and if you're on RedHat/CentOS:
   $ cd /etc/yum.repos.d/
   $ sudo wget -O bigtop.repo
http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/bigtop.repo
   $ yum install 'hadoop*' 'hbase*'

The bad news is that so far we only have HBase compiling against .22
and the Hadoop itself seems to be in need of some tweaks. E.g.:
  https://issues.apache.org/jira/browse/HDFS-2377

That said, after manually patching /usr/lib/hadoop/bin/hdfs and doing:
  # for i in /etc/init.d/hadoop-* ; do $i start ; done

I now have Hadoop 0.22 and HBase 0.92 to play with!

Thanks,
Roman.

I have committed HDFS-1943 to 0.22 so this problem should go away.

Cos

On Tue, Sep 27, 2011 at 05:13PM, Roman Shaposhnik wrote:
> For all those of you who would want to help with testing
> Hadoop 0.22 release, I've setup a job on Bigtop to
> produce an entire Hadoop stack on top of 0.22.
>
> The good news is that it is now super simple to give it
> all a spin. If you are on Ubuntu/Debian you can try:
>    $ cd /etc/apt/sources.list.d
>    $ sudo wget -O bigtop.list
> http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22-ubuntu-lucid/lastSuccessfulBuild/artifact/output/bigtop.list
>    $ sudo apt-get update
>    $ sudo apt-get install 'hadoop*' 'hbase*'
> and if you're on RedHat/CentOS:
>    $ cd /etc/yum.repos.d/
>    $ sudo wget -O bigtop.repo
> http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/bigtop.repo
>    $ yum install 'hadoop*' 'hbase*'
>
> The bad news is that so far we only have HBase compiling against .22
> and the Hadoop itself seems to be in need of some tweaks. E.g.:
>   https://issues.apache.org/jira/browse/HDFS-2377
>
> That said, after manually patching /usr/lib/hadoop/bin/hdfs and doing:
>   # for i in /etc/init.d/hadoop-* ; do $i start ; done
>
> I now have Hadoop 0.22 and HBase 0.92 to play with!
>
> Thanks,
> Roman.

Thanks Roman,

Will have to give BigTop a try...
With the patch for  https://issues.apache.org/jira/browse/PIG-2277 applied
Pig (0.8 for now, and 0.9 in the works) will compile and run against 0.22 as
well.

Thanks,

Joep

________________________________________
From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] On Behalf Of Roman
Shaposhnik [[EMAIL PROTECTED]]
Sent: Tuesday, September 27, 2011 5:13 PM
To: [EMAIL PROTECTED]
Subject: Hadoop 0.22/HBase 0.92 package repos are now available

For all those of you who would want to help with testing
Hadoop 0.22 release, I've setup a job on Bigtop to
produce an entire Hadoop stack on top of 0.22.

The good news is that it is now super simple to give it
all a spin. If you are on Ubuntu/Debian you can try:
   $ cd /etc/apt/sources.list.d
   $ sudo wget -O bigtop.list
http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22-ubuntu-lucid/lastSuccessfulBuild/artifact/output/bigtop.list
   $ sudo apt-get update
   $ sudo apt-get install 'hadoop*' 'hbase*'
and if you're on RedHat/CentOS:
   $ cd /etc/yum.repos.d/
   $ sudo wget -O bigtop.repo
http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22-centos5/lastSuccessfulBuild/artifact/output/bigtop.repo
   $ yum install 'hadoop*' 'hbase*'

The bad news is that so far we only have HBase compiling against .22
and the Hadoop itself seems to be in need of some tweaks. E.g.:
  https://issues.apache.org/jira/browse/HDFS-2377

That said, after manually patching /usr/lib/hadoop/bin/hdfs and doing:
  # for i in /etc/init.d/hadoop-* ; do $i start ; done

I now have Hadoop 0.22 and HBase 0.92 to play with!

Thanks,
Roman.

This is great progress folks !!!

- Milind

On 9/27/11 9:55 PM, "J. Rottinghuis" <[EMAIL PROTECTED]> wrote:

>Thanks Roman,
>
>Will have to give BigTop a try...
>With the patch for  https://issues.apache.org/jira/browse/PIG-2277 applied
>Pig (0.8 for now, and 0.9 in the works) will compile and run against 0.22
>as
>well.
>
>Thanks,
>
>Joep
>
>________________________________________
>From: [EMAIL PROTECTED] [[EMAIL PROTECTED]] On Behalf Of Roman
>Shaposhnik [[EMAIL PROTECTED]]
>Sent: Tuesday, September 27, 2011 5:13 PM
>To: [EMAIL PROTECTED]
>Subject: Hadoop 0.22/HBase 0.92 package repos are now available
>
>For all those of you who would want to help with testing
>Hadoop 0.22 release, I've setup a job on Bigtop to
>produce an entire Hadoop stack on top of 0.22.
>
>The good news is that it is now super simple to give it
>all a spin. If you are on Ubuntu/Debian you can try:
>   $ cd /etc/apt/sources.list.d
>   $ sudo wget -O bigtop.list
>http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22-ubuntu-lucid/lastSuc
>cessfulBuild/artifact/output/bigtop.list
>   $ sudo apt-get update
>   $ sudo apt-get install 'hadoop*' 'hbase*'
>and if you're on RedHat/CentOS:
>   $ cd /etc/yum.repos.d/
>   $ sudo wget -O bigtop.repo
>http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop22-centos5/lastSuccessf
>ulBuild/artifact/output/bigtop.repo
>   $ yum install 'hadoop*' 'hbase*'
>
>The bad news is that so far we only have HBase compiling against .22
>and the Hadoop itself seems to be in need of some tweaks. E.g.:
>  https://issues.apache.org/jira/browse/HDFS-2377
>
>That said, after manually patching /usr/lib/hadoop/bin/hdfs and doing:
>  # for i in /etc/init.d/hadoop-* ; do $i start ; done
>
>I now have Hadoop 0.22 and HBase 0.92 to play with!
>
>Thanks,
>Roman.

On Wed, Sep 28, 2011 at 11:54 AM,  <[EMAIL PROTECTED]> wrote:
> This is great progress folks !!!

Thanks. Actually, for all those who care about .23 I've created a hadoop-0.23
branch in Bigtop and hooked up 3 additional Jenkins jobs:
     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-centos5/
     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-ubuntu-lucid/
     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/

The good news is that HDFS and common are ok, can be packaged and
uses. The bad news for .23 are way more plentiful though:
     1. HBase now breaks with the profile that is supposed to be compatible
     with .23. Perhaps this is one of the issues that Todd has mentioned
     earlier, but I'm not sure.

     2. MR2 (or more precise YARN) seem to be in a pretty weird shape.
     First of all, I can't really build it the same way that the rest of the
     Hadoop 0.23 builds (which is what specified in top-level BUILDING.txt).
     Somehow it requires assembly:single goal to be executed explicitly.
     Worse yet, it seems to be following a different binary layout compared to
     HDFS and common (conf vs. etc/hadoop, etc.). Also, I'm not sure how
     to make natives bits of pipes/utils compile short of manual execution
     of ant.

     3. fuse-dfs is now an orphan. Its not hooked to the build system and seems
     to be in limbo: https://issues.apache.org/jira/browse/HDFS-1757

I would really, really, really appreciate if somebody can help triaging these
issues or at least would be nice enough to point me to existing
JIRAs/discussions.

My goal here is to get us to a point where we can start executing
automated integration testing against a fully distributed cluster and
start publishing these results on the same Jenkins to monitor the
health and progress of .22/.23 and corresponding Bigtop stacks.

Thanks,
Roman.

On Sep 28, 2011, at 8:28 PM, Roman Shaposhnik wrote:

>
>     3. fuse-dfs is now an orphan. Its not hooked to the build system and seems
>     to be in limbo: https://issues.apache.org/jira/browse/HDFS-1757
>

I can volunteer to take a crack at this over the next week - is that a sufficient timeframe?

Brian

On Wed, Sep 28, 2011 at 6:28 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:
> On Wed, Sep 28, 2011 at 11:54 AM,  <[EMAIL PROTECTED]> wrote:
>> This is great progress folks !!!
>
> Thanks. Actually, for all those who care about .23 I've created a hadoop-0.23
> branch in Bigtop and hooked up 3 additional Jenkins jobs:
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-centos5/
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23-ubuntu-lucid/
>     http://bigtop01.cloudera.org:8080/job/Bigtop-hadoop23/
>
> The good news is that HDFS and common are ok, can be packaged and
> uses. The bad news for .23 are way more plentiful though:
>     1. HBase now breaks with the profile that is supposed to be compatible
>     with .23. Perhaps this is one of the issues that Todd has mentioned
>     earlier, but I'm not sure.
>
>     2. MR2 (or more precise YARN) seem to be in a pretty weird shape.
>     First of all, I can't really build it the same way that the rest of the
>     Hadoop 0.23 builds (which is what specified in top-level BUILDING.txt).
>     Somehow it requires assembly:single goal to be executed explicitly.
>     Worse yet, it seems to be following a different binary layout compared to
>     HDFS and common (conf vs. etc/hadoop, etc.). Also, I'm not sure how
>     to make natives bits of pipes/utils compile short of manual execution
>     of ant.
>
>     3. fuse-dfs is now an orphan. Its not hooked to the build system and seems
>     to be in limbo: https://issues.apache.org/jira/browse/HDFS-1757
>
> I would really, really, really appreciate if somebody can help triaging these
> issues or at least would be nice enough to point me to existing
> JIRAs/discussions.

Regarding 2. At the moment you can build MR following the instructions
in the INSTALL file. In fact the nightly build set up by Giri and
Mahadev builds a MR tarball
(https://builds.apache.org/view/G-L/view/Hadoop/job/Hadoop-MapReduce-trunk).
There are some JIRAs to remove the inconsistencies with the rest of
the build, such as MAPREDUCE-3024, MAPREDUCE-3014, HADOOP-7590,
HADOOP-7642. I hope to work on some of these soon, but any help is
always appreciated.

Cheers,
Tom

>
> My goal here is to get us to a point where we can start executing
> automated integration testing against a fully distributed cluster and
> start publishing these results on the same Jenkins to monitor the
> health and progress of .22/.23 and corresponding Bigtop stacks.
>
> Thanks,
> Roman.
>

On Tue, Sep 27, 2011 at 9:55 PM, J. Rottinghuis <[EMAIL PROTECTED]> wrote:
> Will have to give BigTop a try...

Let me know if you have any feedback.

> With the patch for  https://issues.apache.org/jira/browse/PIG-2277 applied
> Pig (0.8 for now, and 0.9 in the works) will compile and run against 0.22 as
> well.

I'm a little bit puzzled by the situation with Pig. It sounds like you need both
Pig 0.8.X and Pig 0.9.X be compilable and usable against .22. Bigtop currently
targets Pig 0.9.0 (since we had troubles compiling Pig 0.8 cleanly)
but if there's
a plan to have a release of Pig 0.8.X that would work with .22 I'd love to
jump on that bandwagon. Any plans on releasing 0.8.2 ?

As for Pig 9 -- will you be pursuing the shims strategy with your patch?

Thanks,
Roman.

Alos, if Pig 0.8.2 or 0.9 is getting released soon it'd critical to have
deploy patch from PIG-1799 getting committed because at the moment Pid doesn't
deploy correct test artifacts to maven.

Cos

P.S. I think this becomes too specific of a discussion and it might find a
better place on one (or more) of the development lists.

On Wed, Sep 28, 2011 at 10:45AM, Roman Shaposhnik wrote:
> On Tue, Sep 27, 2011 at 9:55 PM, J. Rottinghuis <[EMAIL PROTECTED]> wrote:
> > Will have to give BigTop a try...
>
> Let me know if you have any feedback.
>
> > With the patch for ═https://issues.apache.org/jira/browse/PIG-2277 applied
> > Pig (0.8 for now, and 0.9 in the works) will compile and run against 0.22 as
> > well.
>
> I'm a little bit puzzled by the situation with Pig. It sounds like you need both
> Pig 0.8.X and Pig 0.9.X be compilable and usable against .22. Bigtop currently
> targets Pig 0.9.0 (since we had troubles compiling Pig 0.8 cleanly)
> but if there's
> a plan to have a release of Pig 0.8.X that would work with .22 I'd love to
> jump on that bandwagon. Any plans on releasing 0.8.2 ?
>
> As for Pig 9 -- will you be pursuing the shims strategy with your patch?
>
> Thanks,
> Roman.

[CC'ng Pig-dev].

In terms of interest, what is the community more interested in seeing work
with Hadoop 0.22 ? Pig 0.8.x or Pig.0.9.

I took a look at pig 0.9 added features:

New parser (still has corner cases)
UDFs in languages other than Java
Macros
Hcatalog support

As far as existing installations are concerned, my belief is that these
features do not make a very compelling case for pig 0.9 to be compatible
immediately with Hadoop 0.22, and pig 0.8.x makes more sense.

What do people think ?

- Milind

---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)

On 9/29/11 1:53 PM, "Konstantin Boudnik" <[EMAIL PROTECTED]> wrote:

>Alos, if Pig 0.8.2 or 0.9 is getting released soon it'd critical to have
>deploy patch from PIG-1799 getting committed because at the moment Pid
>doesn't
>deploy correct test artifacts to maven.
>
>Cos
>
>P.S. I think this becomes too specific of a discussion and it might find a
>better place on one (or more) of the development lists.
>
>On Wed, Sep 28, 2011 at 10:45AM, Roman Shaposhnik wrote:
>> On Tue, Sep 27, 2011 at 9:55 PM, J. Rottinghuis
>><[EMAIL PROTECTED]> wrote:
>> > Will have to give BigTop a try...
>>
>> Let me know if you have any feedback.
>>
>> > With the patch for �https://issues.apache.org/jira/browse/PIG-2277
>>applied
>> > Pig (0.8 for now, and 0.9 in the works) will compile and run against
>>0.22 as
>> > well.
>>
>> I'm a little bit puzzled by the situation with Pig. It sounds like you
>>need both
>> Pig 0.8.X and Pig 0.9.X be compilable and usable against .22. Bigtop
>>currently
>> targets Pig 0.9.0 (since we had troubles compiling Pig 0.8 cleanly)
>> but if there's
>> a plan to have a release of Pig 0.8.X that would work with .22 I'd love
>>to
>> jump on that bandwagon. Any plans on releasing 0.8.2 ?
>>
>> As for Pig 9 -- will you be pursuing the shims strategy with your patch?
>>
>> Thanks,
>> Roman.
>

2011/9/29  <[EMAIL PROTECTED]>:
> [CC'ng Pig-dev].
>
> In terms of interest, what is the community more interested in seeing work
> with Hadoop 0.22 ? Pig 0.8.x or Pig.0.9.
>
>
> I took a look at pig 0.9 added features:
>
> New parser (still has corner cases)
> UDFs in languages other than Java
> Macros
> Hcatalog support
>
> As far as existing installations are concerned, my belief is that these
> features do not make a very compelling case for pig 0.9 to be compatible
> immediately with Hadoop 0.22, and pig 0.8.x makes more sense.
>
> What do people think ?

Personally, I'd like to see both of these code lines to have a compatibility
releases to address 0.22/0.23 issues. Also, I'd be totally happy with
0.9 NOT addressing this issues if there's 0.10 planned soon enough.

As for 0.8 I really, really, really would like to see a compatibility release.
>From the Bigtop side, I can help with build and testing, but I guess we
need somebody from the Pig community to chime in on how feasible
0.8.2 will be.

Thanks,
Roman.

Roman,

What is your definition of pig 0.8.2 ? Is it 0.8.1 + Hadoop 0.22
compatibility ?

Joep has already made 0.8.1 work with Hadoop 0.22.

- milind

On 9/29/11 3:52 PM, "Roman Shaposhnik" <[EMAIL PROTECTED]> wrote:

>2011/9/29  <[EMAIL PROTECTED]>:
>> [CC'ng Pig-dev].
>>
>> In terms of interest, what is the community more interested in seeing
>>work
>> with Hadoop 0.22 ? Pig 0.8.x or Pig.0.9.
>>
>>
>> I took a look at pig 0.9 added features:
>>
>> New parser (still has corner cases)
>> UDFs in languages other than Java
>> Macros
>> Hcatalog support
>>
>> As far as existing installations are concerned, my belief is that these
>> features do not make a very compelling case for pig 0.9 to be compatible
>> immediately with Hadoop 0.22, and pig 0.8.x makes more sense.
>>
>> What do people think ?
>
>Personally, I'd like to see both of these code lines to have a
>compatibility
>releases to address 0.22/0.23 issues. Also, I'd be totally happy with
>0.9 NOT addressing this issues if there's 0.10 planned soon enough.
>
>As for 0.8 I really, really, really would like to see a compatibility
>release.
>From the Bigtop side, I can help with build and testing, but I guess we
>need somebody from the Pig community to chime in on how feasible
>0.8.2 will be.
>
>Thanks,
>Roman.
>

Pig 8.2 is likely not going to happen (0 interest from any of the
committers).

0.9 has already been released, 9.1 is in the works, as is 10.

Not sure how Milind compiled that list of features in 0.9 but it's kind of
an odd grab-bag of 0.8 and 0.9 features.

The MASSIVE improvement 0.9 introduced is macros. There are other things;
one unheralded in the 3 blog posts below is that handling of nested
structures is cleaned up a lot (in a subtly non-backwards-compatible way).

http://www.hortonworks.com/new-apache-pig-features-part-1-macro/
http://www.hortonworks.com/new-apache-pig-features-part-2-embedding/ (note:
not udfs in other languages, that was already in 8).
http://www.hortonworks.com/new-apache-pig-0-9-features-part-3-additional-features/
On Thu, Sep 29, 2011 at 3:52 PM, Roman Shaposhnik <[EMAIL PROTECTED]> wrote:

> 2011/9/29  <[EMAIL PROTECTED]>:
> > [CC'ng Pig-dev].
> >
> > In terms of interest, what is the community more interested in seeing
> work
> > with Hadoop 0.22 ? Pig 0.8.x or Pig.0.9.
> >
> >
> > I took a look at pig 0.9 added features:
> >
> > New parser (still has corner cases)
> > UDFs in languages other than Java
> > Macros
> > Hcatalog support
> >
> > As far as existing installations are concerned, my belief is that these
> > features do not make a very compelling case for pig 0.9 to be compatible
> > immediately with Hadoop 0.22, and pig 0.8.x makes more sense.
> >
> > What do people think ?
>
> Personally, I'd like to see both of these code lines to have a
> compatibility
> releases to address 0.22/0.23 issues. Also, I'd be totally happy with
> 0.9 NOT addressing this issues if there's 0.10 planned soon enough.
>
> As for 0.8 I really, really, really would like to see a compatibility
> release.
> From the Bigtop side, I can help with build and testing, but I guess we
> need somebody from the Pig community to chime in on how feasible
> 0.8.2 will be.
>
> Thanks,
> Roman.
>

Dmitry,

So, your suggestion would be to try and get 0.9.1 compatible with Hadoop
0.22 ? Would there be an interest from pig committers to see that through,
as a 0.9.2 perhaps ?

- Milind

---
Milind Bhandarkar
Greenplum Labs, EMC
(Disclaimer: Opinions expressed in this email are those of the author, and
do not necessarily represent the views of any organization, past or
present, the author might be affiliated with.)

On 9/29/11 4:30 PM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:

>Pig 8.2 is likely not going to happen (0 interest from any of the
>committers).
>
>0.9 has already been released, 9.1 is in the works, as is 10.
>
>Not sure how Milind compiled that list of features in 0.9 but it's kind of
>an odd grab-bag of 0.8 and 0.9 features.
>
>The MASSIVE improvement 0.9 introduced is macros. There are other things;
>one unheralded in the 3 blog posts below is that handling of nested
>structures is cleaned up a lot (in a subtly non-backwards-compatible way).
>
>http://www.hortonworks.com/new-apache-pig-features-part-1-macro/
>http://www.hortonworks.com/new-apache-pig-features-part-2-embedding/
>(note:
>not udfs in other languages, that was already in 8).
>http://www.hortonworks.com/new-apache-pig-0-9-features-part-3-additional-f
>eatures/
>
>
>On Thu, Sep 29, 2011 at 3:52 PM, Roman Shaposhnik <[EMAIL PROTECTED]>
>wrote:
>
>> 2011/9/29  <[EMAIL PROTECTED]>:
>> > [CC'ng Pig-dev].
>> >
>> > In terms of interest, what is the community more interested in seeing
>> work
>> > with Hadoop 0.22 ? Pig 0.8.x or Pig.0.9.
>> >
>> >
>> > I took a look at pig 0.9 added features:
>> >
>> > New parser (still has corner cases)
>> > UDFs in languages other than Java
>> > Macros
>> > Hcatalog support
>> >
>> > As far as existing installations are concerned, my belief is that
>>these
>> > features do not make a very compelling case for pig 0.9 to be
>>compatible
>> > immediately with Hadoop 0.22, and pig 0.8.x makes more sense.
>> >
>> > What do people think ?
>>
>> Personally, I'd like to see both of these code lines to have a
>> compatibility
>> releases to address 0.22/0.23 issues. Also, I'd be totally happy with
>> 0.9 NOT addressing this issues if there's 0.10 planned soon enough.
>>
>> As for 0.8 I really, really, really would like to see a compatibility
>> release.
>> From the Bigtop side, I can help with build and testing, but I guess we
>> need somebody from the Pig community to chime in on how feasible
>> 0.8.2 will be.
>>
>> Thanks,
>> Roman.
>>

9.1 just went up for vote, and holding up a release to add 0.22
compatibility seems ill-advised.
I would be willing to help get 9.2 out in short order to provide
compatibility with 22, and I think Joep already did a bunch of work to make
this happen.

D

On Thu, Sep 29, 2011 at 4:54 PM, <[EMAIL PROTECTED]> wrote:

> Dmitry,
>
> So, your suggestion would be to try and get 0.9.1 compatible with Hadoop
> 0.22 ? Would there be an interest from pig committers to see that through,
> as a 0.9.2 perhaps ?
>
> - Milind
>
> ---
> Milind Bhandarkar
> Greenplum Labs, EMC
> (Disclaimer: Opinions expressed in this email are those of the author, and
> do not necessarily represent the views of any organization, past or
> present, the author might be affiliated with.)
>
>
>
> On 9/29/11 4:30 PM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:
>
> >Pig 8.2 is likely not going to happen (0 interest from any of the
> >committers).
> >
> >0.9 has already been released, 9.1 is in the works, as is 10.
> >
> >Not sure how Milind compiled that list of features in 0.9 but it's kind of
> >an odd grab-bag of 0.8 and 0.9 features.
> >
> >The MASSIVE improvement 0.9 introduced is macros. There are other things;
> >one unheralded in the 3 blog posts below is that handling of nested
> >structures is cleaned up a lot (in a subtly non-backwards-compatible way).
> >
> >http://www.hortonworks.com/new-apache-pig-features-part-1-macro/
> >http://www.hortonworks.com/new-apache-pig-features-part-2-embedding/
> >(note:
> >not udfs in other languages, that was already in 8).
> >
> http://www.hortonworks.com/new-apache-pig-0-9-features-part-3-additional-f
> >eatures/
> >
> >
> >On Thu, Sep 29, 2011 at 3:52 PM, Roman Shaposhnik <[EMAIL PROTECTED]>
> >wrote:
> >
> >> 2011/9/29  <[EMAIL PROTECTED]>:
> >> > [CC'ng Pig-dev].
> >> >
> >> > In terms of interest, what is the community more interested in seeing
> >> work
> >> > with Hadoop 0.22 ? Pig 0.8.x or Pig.0.9.
> >> >
> >> >
> >> > I took a look at pig 0.9 added features:
> >> >
> >> > New parser (still has corner cases)
> >> > UDFs in languages other than Java
> >> > Macros
> >> > Hcatalog support
> >> >
> >> > As far as existing installations are concerned, my belief is that
> >>these
> >> > features do not make a very compelling case for pig 0.9 to be
> >>compatible
> >> > immediately with Hadoop 0.22, and pig 0.8.x makes more sense.
> >> >
> >> > What do people think ?
> >>
> >> Personally, I'd like to see both of these code lines to have a
> >> compatibility
> >> releases to address 0.22/0.23 issues. Also, I'd be totally happy with
> >> 0.9 NOT addressing this issues if there's 0.10 planned soon enough.
> >>
> >> As for 0.8 I really, really, really would like to see a compatibility
> >> release.
> >> From the Bigtop side, I can help with build and testing, but I guess we
> >> need somebody from the Pig community to chime in on how feasible
> >> 0.8.2 will be.
> >>
> >> Thanks,
> >> Roman.
> >>
>
>

Excellent. 0.9.2 it is, then ! Thanks @squarecog.

- milind
On 9/30/11 11:26 AM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:

>9.1 just went up for vote, and holding up a release to add 0.22
>compatibility seems ill-advised.
>I would be willing to help get 9.2 out in short order to provide
>compatibility with 22, and I think Joep already did a bunch of work to
>make
>this happen.
>
>D
>
>On Thu, Sep 29, 2011 at 4:54 PM, <[EMAIL PROTECTED]> wrote:
>
>> Dmitry,
>>
>> So, your suggestion would be to try and get 0.9.1 compatible with Hadoop
>> 0.22 ? Would there be an interest from pig committers to see that
>>through,
>> as a 0.9.2 perhaps ?
>>
>> - Milind
>>
>> ---
>> Milind Bhandarkar
>> Greenplum Labs, EMC
>> (Disclaimer: Opinions expressed in this email are those of the author,
>>and
>> do not necessarily represent the views of any organization, past or
>> present, the author might be affiliated with.)
>>
>>
>>
>> On 9/29/11 4:30 PM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:
>>
>> >Pig 8.2 is likely not going to happen (0 interest from any of the
>> >committers).
>> >
>> >0.9 has already been released, 9.1 is in the works, as is 10.
>> >
>> >Not sure how Milind compiled that list of features in 0.9 but it's
>>kind of
>> >an odd grab-bag of 0.8 and 0.9 features.
>> >
>> >The MASSIVE improvement 0.9 introduced is macros. There are other
>>things;
>> >one unheralded in the 3 blog posts below is that handling of nested
>> >structures is cleaned up a lot (in a subtly non-backwards-compatible
>>way).
>> >
>> >http://www.hortonworks.com/new-apache-pig-features-part-1-macro/
>> >http://www.hortonworks.com/new-apache-pig-features-part-2-embedding/
>> >(note:
>> >not udfs in other languages, that was already in 8).
>> >
>>
>>http://www.hortonworks.com/new-apache-pig-0-9-features-part-3-additional-
>>f
>> >eatures/
>> >
>> >
>> >On Thu, Sep 29, 2011 at 3:52 PM, Roman Shaposhnik <[EMAIL PROTECTED]>
>> >wrote:
>> >
>> >> 2011/9/29  <[EMAIL PROTECTED]>:
>> >> > [CC'ng Pig-dev].
>> >> >
>> >> > In terms of interest, what is the community more interested in
>>seeing
>> >> work
>> >> > with Hadoop 0.22 ? Pig 0.8.x or Pig.0.9.
>> >> >
>> >> >
>> >> > I took a look at pig 0.9 added features:
>> >> >
>> >> > New parser (still has corner cases)
>> >> > UDFs in languages other than Java
>> >> > Macros
>> >> > Hcatalog support
>> >> >
>> >> > As far as existing installations are concerned, my belief is that
>> >>these
>> >> > features do not make a very compelling case for pig 0.9 to be
>> >>compatible
>> >> > immediately with Hadoop 0.22, and pig 0.8.x makes more sense.
>> >> >
>> >> > What do people think ?
>> >>
>> >> Personally, I'd like to see both of these code lines to have a
>> >> compatibility
>> >> releases to address 0.22/0.23 issues. Also, I'd be totally happy with
>> >> 0.9 NOT addressing this issues if there's 0.10 planned soon enough.
>> >>
>> >> As for 0.8 I really, really, really would like to see a compatibility
>> >> release.
>> >> From the Bigtop side, I can help with build and testing, but I guess
>>we
>> >> need somebody from the Pig community to chime in on how feasible
>> >> 0.8.2 will be.
>> >>
>> >> Thanks,
>> >> Roman.
>> >>
>>
>>