Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Pig >> mail # user >> How can I set the mapper number for pig script?


Copy link to this message
-
Re: How can I set the mapper number for pig script?
Thanks for all your help.

My pig script may have some cpu-intensive job like nlp processing, so it
would be helpful if I have multiple mappers running. Correct me if I am
wrong.
Thanks,

Sheng

On Sat, Jun 23, 2012 at 9:40 AM, Scott Foster <[EMAIL PROTECTED]>wrote:

> You can also turn off split combination completely and then the number
> of mappers will equal the number of blocks
> SET pig.noSplitCombination false;
>
> Adding mappers may not make your process run faster since the time to
> read the data may be less than the overhead of creating a new JVM for
> each map task.
>
> scott.
>