I was just working on a pig script to group some data by a field and then generate percentages for each group.  Without windowing functions at my disposal,  I wound up using a group by on the field for the numerator, a group all for the denominator, and a cross to pull them together.

Afterward, I found myself wanting to drop that snippet somewhere that other folks could use it, or (even better) to have found a snippet in the first place and not had to write it!

So I was thinking it'd be great to have a "pig cookbook" online with common operations that people do with pig.  Does anything like that exist yet?  If not, where would be a good place for it?

Best,
Doug

Sounds like a good candidate for Piggybank for Macro. We don't have it yet
and hopefully we can have it soon.

Currently you can drop your code to FAQ (
https://cwiki.apache.org/confluence/display/PIG/FAQ). If we have
significant number of recipe, we can move them to a separate section in
cwiki.

Daniel

On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels <[EMAIL PROTECTED]>wrote:

> I was just working on a pig script to group some data by a field and then
> generate percentages for each group.  Without windowing functions at my
> disposal,  I wound up using a group by on the field for the numerator, a
> group all for the denominator, and a cross to pull them together.
>
> Afterward, I found myself wanting to drop that snippet somewhere that
> other folks could use it, or (even better) to have found a snippet in the
> first place and not had to write it!
>
> So I was thinking it'd be great to have a "pig cookbook" online with
> common operations that people do with pig.  Does anything like that exist
> yet?  If not, where would be a good place for it?
>
> Best,
> Doug
>

We used to have a cookbook.
http://pig.apache.org/docs/r0.7.0/cookbook.htmlBut cant find similar
link for 0.8/0.9

Ashutosh
On Thu, Nov 10, 2011 at 00:34, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Sounds like a good candidate for Piggybank for Macro. We don't have it yet
> and hopefully we can have it soon.
>
> Currently you can drop your code to FAQ (
> https://cwiki.apache.org/confluence/display/PIG/FAQ). If we have
> significant number of recipe, we can move them to a separate section in
> cwiki.
>
> Daniel
>
> On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels <[EMAIL PROTECTED]
> >wrote:
>
> > I was just working on a pig script to group some data by a field and then
> > generate percentages for each group.  Without windowing functions at my
> > disposal,  I wound up using a group by on the field for the numerator, a
> > group all for the denominator, and a cross to pull them together.
> >
> > Afterward, I found myself wanting to drop that snippet somewhere that
> > other folks could use it, or (even better) to have found a snippet in the
> > first place and not had to write it!
> >
> > So I was thinking it'd be great to have a "pig cookbook" online with
> > common operations that people do with pig.  Does anything like that exist
> > yet?  If not, where would be a good place for it?
> >
> > Best,
> > Doug
> >
>

In Pig 9 this material was moved into Performance and Efficiency, section Performance Enhancers:

http://pig.apache.org/docs/r0.9.1/perf.html#performance-enhancers

Thanks/C
-----Original Message-----
From: Ashutosh Chauhan [mailto:[EMAIL PROTECTED]]
Sent: Thursday, November 10, 2011 11:00 AM
To: [EMAIL PROTECTED]
Subject: Re: pig cookbook

We used to have a cookbook.
http://pig.apache.org/docs/r0.7.0/cookbook.htmlBut cant find similar
link for 0.8/0.9

Ashutosh
On Thu, Nov 10, 2011 at 00:34, Daniel Dai <[EMAIL PROTECTED]> wrote:

> Sounds like a good candidate for Piggybank for Macro. We don't have it yet
> and hopefully we can have it soon.
>
> Currently you can drop your code to FAQ (
> https://cwiki.apache.org/confluence/display/PIG/FAQ). If we have
> significant number of recipe, we can move them to a separate section in
> cwiki.
>
> Daniel
>
> On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels <[EMAIL PROTECTED]
> >wrote:
>
> > I was just working on a pig script to group some data by a field and then
> > generate percentages for each group.  Without windowing functions at my
> > disposal,  I wound up using a group by on the field for the numerator, a
> > group all for the denominator, and a cross to pull them together.
> >
> > Afterward, I found myself wanting to drop that snippet somewhere that
> > other folks could use it, or (even better) to have found a snippet in the
> > first place and not had to write it!
> >
> > So I was thinking it'd be great to have a "pig cookbook" online with
> > common operations that people do with pig.  Does anything like that exist
> > yet?  If not, where would be a good place for it?
> >
> > Best,
> > Doug
> >
>

Wait, a cross? No, don't do a cross.  You can just cast the result of group
all as a scalar, this will be much more efficient.

D

On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels <[EMAIL PROTECTED]>wrote:

> I was just working on a pig script to group some data by a field and then
> generate percentages for each group.  Without windowing functions at my
> disposal,  I wound up using a group by on the field for the numerator, a
> group all for the denominator, and a cross to pull them together.
>
> Afterward, I found myself wanting to drop that snippet somewhere that
> other folks could use it, or (even better) to have found a snippet in the
> first place and not had to write it!
>
> So I was thinking it'd be great to have a "pig cookbook" online with
> common operations that people do with pig.  Does anything like that exist
> yet?  If not, where would be a good place for it?
>
> Best,
> Doug
>

Thanks Dmitriy, I didn't know you could do that!

I added an entry to the FAQ that Daniel pointed out: http://bit.ly/sZ8UOL

-Doug

On 11/10/11 6:25 PM, "Dmitriy Ryaboy" <[EMAIL PROTECTED]> wrote:

>Wait, a cross? No, don't do a cross.  You can just cast the result of
>group
>all as a scalar, this will be much more efficient.
>
>D
>
>On Wed, Nov 9, 2011 at 6:32 PM, Doug Daniels
><[EMAIL PROTECTED]>wrote:
>
>> I was just working on a pig script to group some data by a field and
>>then
>> generate percentages for each group.  Without windowing functions at my
>> disposal,  I wound up using a group by on the field for the numerator, a
>> group all for the denominator, and a cross to pull them together.
>>
>> Afterward, I found myself wanting to drop that snippet somewhere that
>> other folks could use it, or (even better) to have found a snippet in
>>the
>> first place and not had to write it!
>>
>> So I was thinking it'd be great to have a "pig cookbook" online with
>> common operations that people do with pig.  Does anything like that
>>exist
>> yet?  If not, where would be a good place for it?
>>
>> Best,
>> Doug
>>