Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> How to coalesce fields in Pig?


+
James Kebinger 2011-08-29, 18:15
+
Dmitriy Ryaboy 2011-08-29, 18:29
Copy link to this message
-
Re: How to coalesce fields in Pig?
Thanks, it must have been the lack of parenthesis that did me in when i
tried the ternary expression, or some other typo. I'll use that in the
future.

On Mon, Aug 29, 2011 at 2:29 PM, Dmitriy Ryaboy <[EMAIL PROTECTED]> wrote:

> Hi James,
> I use ternary expressions for this: foreach joined generate ( rel1.x is
> null
> ? rel2.x : rel1.x) as x;
>
> On Mon, Aug 29, 2011 at 11:15 AM, James Kebinger <[EMAIL PROTECTED]
> >wrote:
>
> > My apologies if this is in the docs somewhere, I was unable to find
> > anything, but I might be calling it the wrong name.
> >
> > I'm doing a full outer join in Pig - as such, one or the other join keys
> > may
> > be null. I'd like to be able to look at 2 columns, and retrieve just the
> > one
> > that is not null. Is that possible?
> >
> > I tried an expression in generate with is null and the ternary operator,
> > and
> > took a look at DECODE. That might do the trick but wasn't sure if null
> > checking would work, and if other expressions could appear inside the
> > decode.
> >
> > In my case the fields are integers, so I abused the MAX and TOBAG
> operators
> > like this MAX(TOBAG(rx_keyed::u2,cx_keyed::u2)) to get the effect I was
> > after, but I would love to know if there's a better way.
> >
> > Thanks for your time!
> >
> > -James Kebinger
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB