Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Conditional execution of 'generate' clauses


+
Stan Rosenberg 2011-10-02, 16:22
+
Stan Rosenberg 2011-10-02, 16:28
+
Dmitriy Ryaboy 2011-10-03, 07:46
Copy link to this message
-
Re: Conditional execution of 'generate' clauses
I am aware of workarounds using filters.  However, a filter must be executed
unconditionally whereas with conditional control-flow
some (generate) clauses may never need to be executed.

Thanks,

stan

On Mon, Oct 3, 2011 at 3:46 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Why not this:
>
> Y = foreach X { ..
>  t = distinct ...
>  generate t, foo...
> }
>
> Z = filter Y by isEmpty(t);
>
> OR: t can't be empty if the thing you are distincting is not empty, so this
> should work:
>
> Y = filter X by IsEmpty(thing_you_wanted_to_distinct);
> Z = foreach Y {
>  -- the thing you are distincting is now guaranteed to have at least 1
> value
>  t = distinct ..
>  generate foo...
> }
>
> On Sun, Oct 2, 2011 at 9:28 AM, Stan Rosenberg <
> [EMAIL PROTECTED]> wrote:
>
> > Hi Folks,
> >
> > I came across a use case where I'd like to do something like this:
> >
> > FOREACH X {
> >    ...
> >   t = DISTINCT (...)
> >   if (!IsEmpty(t))
> >      GENERATE foo, ...
> > }
> >
> > Thus, 'generate' is conditionally executed and the control flow depends
> on
> > the value of some tuple 't'.
> > Can this be done in pig?
> >
> > Thanks,
> >
> > stan
> >
> > P.S. Please ignore my previous email; I accidentally triggered send
> before
> > I
> > had a chance to finish it.
> >
>
+
Alan Gates 2011-10-03, 15:42
+
Stan Rosenberg 2011-10-03, 18:18
+
Alan Gates 2011-10-03, 18:33