Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Conditional execution of 'generate' clauses


+
Stan Rosenberg 2011-10-02, 16:22
+
Stan Rosenberg 2011-10-02, 16:28
+
Dmitriy Ryaboy 2011-10-03, 07:46
Copy link to this message
-
Re: Conditional execution of 'generate' clauses
I am aware of workarounds using filters.  However, a filter must be executed
unconditionally whereas with conditional control-flow
some (generate) clauses may never need to be executed.

Thanks,

stan

On Mon, Oct 3, 2011 at 3:46 AM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Why not this:
>
> Y = foreach X { ..
>  t = distinct ...
>  generate t, foo...
> }
>
> Z = filter Y by isEmpty(t);
>
> OR: t can't be empty if the thing you are distincting is not empty, so this
> should work:
>
> Y = filter X by IsEmpty(thing_you_wanted_to_distinct);
> Z = foreach Y {
>  -- the thing you are distincting is now guaranteed to have at least 1
> value
>  t = distinct ..
>  generate foo...
> }
>
> On Sun, Oct 2, 2011 at 9:28 AM, Stan Rosenberg <
> [EMAIL PROTECTED]> wrote:
>
> > Hi Folks,
> >
> > I came across a use case where I'd like to do something like this:
> >
> > FOREACH X {
> >    ...
> >   t = DISTINCT (...)
> >   if (!IsEmpty(t))
> >      GENERATE foo, ...
> > }
> >
> > Thus, 'generate' is conditionally executed and the control flow depends
> on
> > the value of some tuple 't'.
> > Can this be done in pig?
> >
> > Thanks,
> >
> > stan
> >
> > P.S. Please ignore my previous email; I accidentally triggered send
> before
> > I
> > had a chance to finish it.
> >
>
+
Alan Gates 2011-10-03, 15:42
+
Stan Rosenberg 2011-10-03, 18:18
+
Alan Gates 2011-10-03, 18:33
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB