Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> how to use option mapred.max.map.failures.percent ?


+
Wojciech Langiewicz 2010-09-28, 08:49
+
Renato Marroquín Mogrovej... 2010-09-30, 23:24
Copy link to this message
-
Re: how to use option mapred.max.map.failures.percent ?

To set heap size you can set the java property when you invoke pig-
-Dmapred.child.java.opts=-Xmx2048m

This should work for mapred.max.map.failures.percent as well.

-Thejas
On 9/30/10 4:24 PM, "Renato Marroquín Mogrovejo" <[EMAIL PROTECTED]> wrote:

Hi all, I'm also having a similar problem, does anyone have a
suggestion/opinion/thought?
The thing is that my MR jobs are dying because of java heap space error (I
am planning to get some more). I want a way to control this, I mean this
parameter would also help me handling this memory.
Renato M.
2010/9/28 Wojciech Langiewicz <[EMAIL PROTECTED]>

> Hello,
> I'm running very big MR job with Pig, and sometimes some maps fail, but I
> would like this job to finish anyway. I know that option
> "mapred.max.map.failures.percent" is what I need, but how to pass this
> option when running Pig? (from command line or interpreter)
>
> Thanks for help.
>
> --
> Wojciech Langiewicz
>
+
Wojciech Langiewicz 2010-10-06, 22:02
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB