-Re: pig generated 2 map-only jobs ?
Yang 2012-06-17, 04:06
I attached the trimmed version of my script .
basically the similars var generates a bag, explodes it, after that, each
of the output record is filtered through a Udf.
I suspect that the 2 maps are due to the explosion. but it should be
possible to put the above sequence into a single map.
On Tue, Jun 12, 2012 at 2:14 PM, Alan Gates <[EMAIL PROTECTED]> wrote:
> There are cases where it would do this, such as unioning two inputs. Can
> you send your script to the list?
> On Jun 11, 2012, at 11:21 PM, Yang wrote:
> > this is what happened with my pig script.
> > why would it generate 2 map-only jobs?
> > wouldn't the optimization process chain together both mappers and keep
> > 1 mapper stage?
> > thanks
> > Yang