Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Conditional execution of 'generate' clauses


+
Stan Rosenberg 2011-10-02, 16:22
+
Stan Rosenberg 2011-10-02, 16:28
+
Dmitriy Ryaboy 2011-10-03, 07:46
+
Stan Rosenberg 2011-10-03, 14:16
+
Alan Gates 2011-10-03, 15:42
Copy link to this message
-
Re: Conditional execution of 'generate' clauses
Alan,

Let me abstract my previous example to:

foreach X {
   -- do some processing and store results in Y
   if (Y.$0 == 'foo') {
      generate X.$0, ...
   }
}

Does pig support this type of control-flow?

Many thanks,

stan

On Mon, Oct 3, 2011 at 11:42 AM, Alan Gates <[EMAIL PROTECTED]> wrote:

> I'm not sure what you mean by t being non empty, since it's a relation and
> not an expression.  But guessing that you mean there is some bag in t you
> want to check for non-emptiness, isn't the following equivalent?
>
> foreach X {
>    ...
>    t = distinct ...
>    tnotempty = filter t by !IsEmpty(...);
>    generate foo, ...
> }
>
> Alan.
>
>
> On Oct 3, 2011, at 12:46 AM, Dmitriy Ryaboy wrote:
>
> > Why not this:
> >
> > Y = foreach X { ..
> >  t = distinct ...
> >  generate t, foo...
> > }
> >
> > Z = filter Y by isEmpty(t);
> >
> > OR: t can't be empty if the thing you are distincting is not empty, so
> this
> > should work:
> >
> > Y = filter X by IsEmpty(thing_you_wanted_to_distinct);
> > Z = foreach Y {
> >  -- the thing you are distincting is now guaranteed to have at least 1
> > value
> >  t = distinct ..
> >  generate foo...
> > }
> >
> > On Sun, Oct 2, 2011 at 9:28 AM, Stan Rosenberg <
> > [EMAIL PROTECTED]> wrote:
> >
> >> Hi Folks,
> >>
> >> I came across a use case where I'd like to do something like this:
> >>
> >> FOREACH X {
> >>   ...
> >>  t = DISTINCT (...)
> >>  if (!IsEmpty(t))
> >>     GENERATE foo, ...
> >> }
> >>
> >> Thus, 'generate' is conditionally executed and the control flow depends
> on
> >> the value of some tuple 't'.
> >> Can this be done in pig?
> >>
> >> Thanks,
> >>
> >> stan
> >>
> >> P.S. Please ignore my previous email; I accidentally triggered send
> before
> >> I
> >> had a chance to finish it.
> >>
>
>
+
Alan Gates 2011-10-03, 18:33