Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - How to coalesce fields in Pig?


Copy link to this message
-
How to coalesce fields in Pig?
James Kebinger 2011-08-29, 18:15
My apologies if this is in the docs somewhere, I was unable to find
anything, but I might be calling it the wrong name.

I'm doing a full outer join in Pig - as such, one or the other join keys may
be null. I'd like to be able to look at 2 columns, and retrieve just the one
that is not null. Is that possible?

I tried an expression in generate with is null and the ternary operator, and
took a look at DECODE. That might do the trick but wasn't sure if null
checking would work, and if other expressions could appear inside the
decode.

In my case the fields are integers, so I abused the MAX and TOBAG operators
like this MAX(TOBAG(rx_keyed::u2,cx_keyed::u2)) to get the effect I was
after, but I would love to know if there's a better way.

Thanks for your time!

-James Kebinger