Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig, mail # user - pig generated 2 map-only jobs ?


Copy link to this message
-
Re: pig generated 2 map-only jobs ?
Alan Gates 2012-06-17, 06:51
Apache mailing lists strip all attachments.  You'll have to inline the script in your message or post it somewhere and send a link.

Alan.

On Jun 16, 2012, at 9:06 PM, Yang wrote:

> Thanks Alan.
>
>
> I attached the trimmed version of my script .
>
>
> basically the similars var generates a bag, explodes it, after that, each of the output record is filtered through a Udf.
>
> I suspect that the 2 maps are due to the explosion. but it should be possible to put the above sequence into a single map.
>
>
> Yang
>
> On Tue, Jun 12, 2012 at 2:14 PM, Alan Gates <[EMAIL PROTECTED]> wrote:
> There are cases where it would do this, such as unioning two inputs.  Can you send your script to the list?
>
> Alan.
>
> On Jun 11, 2012, at 11:21 PM, Yang wrote:
>
> > this is what happened with my pig script.
> > why would it generate 2 map-only jobs?
> > wouldn't the optimization process chain together both mappers and keep only
> > 1 mapper stage?
> >
> >
> > thanks
> > Yang
>
>