Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> pig generated 2 map-only jobs ?


Copy link to this message
-
Re: pig generated 2 map-only jobs ?
Apache mailing lists strip all attachments.  You'll have to inline the script in your message or post it somewhere and send a link.

Alan.

On Jun 16, 2012, at 9:06 PM, Yang wrote:

> Thanks Alan.
>
>
> I attached the trimmed version of my script .
>
>
> basically the similars var generates a bag, explodes it, after that, each of the output record is filtered through a Udf.
>
> I suspect that the 2 maps are due to the explosion. but it should be possible to put the above sequence into a single map.
>
>
> Yang
>
> On Tue, Jun 12, 2012 at 2:14 PM, Alan Gates <[EMAIL PROTECTED]> wrote:
> There are cases where it would do this, such as unioning two inputs.  Can you send your script to the list?
>
> Alan.
>
> On Jun 11, 2012, at 11:21 PM, Yang wrote:
>
> > this is what happened with my pig script.
> > why would it generate 2 map-only jobs?
> > wouldn't the optimization process chain together both mappers and keep only
> > 1 mapper stage?
> >
> >
> > thanks
> > Yang
>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB