Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Pig >> mail # user >> how to use option mapred.max.map.failures.percent ?


+
Wojciech Langiewicz 2010-09-28, 08:49
+
Renato Marroquín Mogrovej... 2010-09-30, 23:24
Copy link to this message
-
Re: how to use option mapred.max.map.failures.percent ?

To set heap size you can set the java property when you invoke pig-
-Dmapred.child.java.opts=-Xmx2048m

This should work for mapred.max.map.failures.percent as well.

-Thejas
On 9/30/10 4:24 PM, "Renato Marroquín Mogrovejo" <[EMAIL PROTECTED]> wrote:

Hi all, I'm also having a similar problem, does anyone have a
suggestion/opinion/thought?
The thing is that my MR jobs are dying because of java heap space error (I
am planning to get some more). I want a way to control this, I mean this
parameter would also help me handling this memory.
Renato M.
2010/9/28 Wojciech Langiewicz <[EMAIL PROTECTED]>

> Hello,
> I'm running very big MR job with Pig, and sometimes some maps fail, but I
> would like this job to finish anyway. I know that option
> "mapred.max.map.failures.percent" is what I need, but how to pass this
> option when running Pig? (from command line or interpreter)
>
> Thanks for help.
>
> --
> Wojciech Langiewicz
>
+
Wojciech Langiewicz 2010-10-06, 22:02