Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> How can I set the mapper number for pig script?


+
Sheng Guo 2012-06-23, 02:27
+
Jagat Singh 2012-06-23, 04:31
+
Sheng Guo 2012-06-23, 07:30
+
Stan Rosenberg 2012-06-23, 15:13
+
Scott Foster 2012-06-23, 16:40
Copy link to this message
-
Re: How can I set the mapper number for pig script?
Thanks for all your help.

My pig script may have some cpu-intensive job like nlp processing, so it
would be helpful if I have multiple mappers running. Correct me if I am
wrong.
Thanks,

Sheng

On Sat, Jun 23, 2012 at 9:40 AM, Scott Foster <[EMAIL PROTECTED]>wrote:

> You can also turn off split combination completely and then the number
> of mappers will equal the number of blocks
> SET pig.noSplitCombination false;
>
> Adding mappers may not make your process run faster since the time to
> read the data may be less than the overhead of creating a new JVM for
> each map task.
>
> scott.
>
+
Yang 2012-06-23, 21:58
+
John Meagher 2012-06-23, 23:15
+
Scott Foster 2012-06-26, 23:47
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB