Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> How can I set the mapper number for pig script?


+
Sheng Guo 2012-06-23, 02:27
+
Jagat Singh 2012-06-23, 04:31
+
Sheng Guo 2012-06-23, 07:30
+
Stan Rosenberg 2012-06-23, 15:13
+
Scott Foster 2012-06-23, 16:40
Copy link to this message
-
Re: How can I set the mapper number for pig script?
Thanks for all your help.

My pig script may have some cpu-intensive job like nlp processing, so it
would be helpful if I have multiple mappers running. Correct me if I am
wrong.
Thanks,

Sheng

On Sat, Jun 23, 2012 at 9:40 AM, Scott Foster <[EMAIL PROTECTED]>wrote:

> You can also turn off split combination completely and then the number
> of mappers will equal the number of blocks
> SET pig.noSplitCombination false;
>
> Adding mappers may not make your process run faster since the time to
> read the data may be less than the overhead of creating a new JVM for
> each map task.
>
> scott.
>
+
Yang 2012-06-23, 21:58
+
John Meagher 2012-06-23, 23:15
+
Scott Foster 2012-06-26, 23:47