Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - this nested foreach worked in 0.8.1, no longer works in 0.10


Copy link to this message
-
Re: this nested foreach worked in 0.8.1, no longer works in 0.10
Dmitriy Ryaboy 2012-06-07, 23:06
Yeah, anything that reproduces the error would be great. Thanks!

D

On Thu, Jun 7, 2012 at 10:35 AM, Steve Bernstein
<[EMAIL PROTECTED]> wrote:
> Thanks Dmitriy.  The full script is probably more than you want (and more than I think I'd want to send), but let me create a "subset" of it, run it to check that it's otherwise sound and that the error replicates, then send that.  Would that be okay?
>
> -----Original Message-----
> From: Dmitriy Ryaboy [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, June 06, 2012 11:56 AM
> To: [EMAIL PROTECTED]
> Subject: Re: this nested foreach worked in 0.8.1, no longer works in 0.10
>
> Any chance you can send the full script?
>
> On Jun 6, 2012, at 11:44 AM, Steve Bernstein <[EMAIL PROTECTED]> wrote:
>
>> Meanwhile, I'm moving forward running this script on 0.8.1 with no issues.  Probably going to back off on using 0.10 for now and wait for a more stable release.
>>
>> -----Original Message-----
>> From: Steve Bernstein [mailto:[EMAIL PROTECTED]]
>> Sent: Sunday, June 03, 2012 9:25 AM
>> To: [EMAIL PROTECTED]
>> Subject: RE: this nested foreach worked in 0.8.1, no longer works in
>> 0.10
>>
>> Since this seems to do with project, before I wrote to the list, I'd already tried actionBag.sent and actionBagGrouped::actionBag.sent.  No joy.  Is this a bug?  Seems like others would have stumbled across it already.
>> SB
>> ________________________________________
>> From: Subir S [[EMAIL PROTECTED]]
>> Sent: Saturday, June 02, 2012 4:04 AM
>> To: [EMAIL PROTECTED]
>> Subject: Re: this nested foreach worked in 0.8.1, no longer works in
>> 0.10
>>
>> Can you try  sent = FILTER actionBag BY  actionBag.action=='sent';
>>
>> Although not sure why it does not work. I also use pig 0.8.1 and similar constructs work for me.
>>
>> On Fri, Jun 1, 2012 at 10:55 PM, Steve Bernstein
>> <[EMAIL PROTECTED]>wrote:
>>
>>> No, all those relations embedded in the nested foreach are unique to
>>> it, not used elsewhere.
>>> SB
>>>
>>> -----Original Message-----
>>> From: Dmitriy Ryaboy [mailto:[EMAIL PROTECTED]]
>>> Sent: Friday, June 01, 2012 2:03 AM
>>> To: [EMAIL PROTECTED]
>>> Cc: [EMAIL PROTECTED]
>>> Subject: Re: this nested foreach worked in 0.8.1, no longer works in
>>> 0.10
>>>
>>> Do you have relations ned sent, opened, etc, elsewhere in the script?
>>>
>>> On May 31, 2012, at 12:09 PM, Steve Bernstein
>>> <[EMAIL PROTECTED]>
>>> wrote:
>>>
>>>> We upgraded from Pig 0.8.1 to 0.10 and the following nested foreach
>>>> no
>>> longer works:
>>>>
>>>> actionBagGrouped = GROUP actionBag BY (deal_id,month); dealCounts >>>> FOREACH actionBagGrouped {
>>>>               sent = FILTER actionBag BY action=='sent';
>>>>               opened = FILTER actionBag BY action=='opened';
>>>>               clicked = FILTER actionBag BY action=='clicked';
>>>>               purchased = FILTER actionBag BY action=='purchased';
>>>>
>>>>               GENERATE
>>>>                               FLATTEN(group),
>>>>                               COUNT(sent) AS sentCount,
>>>>                               COUNT(opened) AS openedCount,
>>>>                               COUNT(clicked) AS clickedCount,
>>>>                               COUNT(purchased) AS purchasedCount; };
>>>> I get:
>>>> "Invalid scalar projection: action : A column needs to be projected
>>>> from
>>> a relation for it to be used as a scalar"
>>>> Banging my head against a wall-the problem is with the nest op sent
>>>> >>> Filter...
>>>>
>>>> Help!
>>>> Thanks-
>>>> Steve
>>>