Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - pig generated 2 map-only jobs ?


Copy link to this message
-
Re: pig generated 2 map-only jobs ?
Yang 2012-06-17, 04:06
Thanks Alan.
I attached the trimmed version of my script .
basically the similars var generates a bag, explodes it, after that, each
of the output record is filtered through a Udf.

I suspect that the 2 maps are due to the explosion. but it should be
possible to put the above sequence into a single map.
Yang

On Tue, Jun 12, 2012 at 2:14 PM, Alan Gates <[EMAIL PROTECTED]> wrote:

> There are cases where it would do this, such as unioning two inputs.  Can
> you send your script to the list?
>
> Alan.
>
> On Jun 11, 2012, at 11:21 PM, Yang wrote:
>
> > this is what happened with my pig script.
> > why would it generate 2 map-only jobs?
> > wouldn't the optimization process chain together both mappers and keep
> only
> > 1 mapper stage?
> >
> >
> > thanks
> > Yang
>
>