Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> Conditional execution of 'generate' clauses


+
Stan Rosenberg 2011-10-02, 16:22
+
Stan Rosenberg 2011-10-02, 16:28
+
Dmitriy Ryaboy 2011-10-03, 07:46
+
Stan Rosenberg 2011-10-03, 14:16
+
Alan Gates 2011-10-03, 15:42
Copy link to this message
-
Re: Conditional execution of 'generate' clauses
Alan,

Let me abstract my previous example to:

foreach X {
   -- do some processing and store results in Y
   if (Y.$0 == 'foo') {
      generate X.$0, ...
   }
}

Does pig support this type of control-flow?

Many thanks,

stan

On Mon, Oct 3, 2011 at 11:42 AM, Alan Gates <[EMAIL PROTECTED]> wrote:

> I'm not sure what you mean by t being non empty, since it's a relation and
> not an expression.  But guessing that you mean there is some bag in t you
> want to check for non-emptiness, isn't the following equivalent?
>
> foreach X {
>    ...
>    t = distinct ...
>    tnotempty = filter t by !IsEmpty(...);
>    generate foo, ...
> }
>
> Alan.
>
>
> On Oct 3, 2011, at 12:46 AM, Dmitriy Ryaboy wrote:
>
> > Why not this:
> >
> > Y = foreach X { ..
> >  t = distinct ...
> >  generate t, foo...
> > }
> >
> > Z = filter Y by isEmpty(t);
> >
> > OR: t can't be empty if the thing you are distincting is not empty, so
> this
> > should work:
> >
> > Y = filter X by IsEmpty(thing_you_wanted_to_distinct);
> > Z = foreach Y {
> >  -- the thing you are distincting is now guaranteed to have at least 1
> > value
> >  t = distinct ..
> >  generate foo...
> > }
> >
> > On Sun, Oct 2, 2011 at 9:28 AM, Stan Rosenberg <
> > [EMAIL PROTECTED]> wrote:
> >
> >> Hi Folks,
> >>
> >> I came across a use case where I'd like to do something like this:
> >>
> >> FOREACH X {
> >>   ...
> >>  t = DISTINCT (...)
> >>  if (!IsEmpty(t))
> >>     GENERATE foo, ...
> >> }
> >>
> >> Thus, 'generate' is conditionally executed and the control flow depends
> on
> >> the value of some tuple 't'.
> >> Can this be done in pig?
> >>
> >> Thanks,
> >>
> >> stan
> >>
> >> P.S. Please ignore my previous email; I accidentally triggered send
> before
> >> I
> >> had a chance to finish it.
> >>
>
>
+
Alan Gates 2011-10-03, 18:33
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB