-Re: Avoid Killing of YARN container on excess virtual memory usage
Krishna Kishore Bonagiri 2012-10-19, 01:47
Thanks Harsha, I was actually looking this part of the code in
ContainerMonitorImpl.java trying to find how can I make use of it, but as
you said it's a bug and doesn't seem to be really handling the case.
On Thu, Oct 18, 2012 at 5:07 PM, Harsh J <[EMAIL PROTECTED]> wrote:
> This is possible to do, but you've hit a bug with the current YARN
> implementation. Ideally you should be able to configure the vmem-pmem
> ratio (or an equivalent config) to be -1, to indicate disabling of
> virtual memory checks completely (and there's indeed checks for this),
> but it seems like we are enforcing the ratio to be at least 1.0 (and
> hence negatives are disallowed).
> You can't workaround by setting the NM's offered resource.mb to -1
> either, as you'll lose out on controlling maximum allocations.
> Please file a YARN bug on JIRA. The code at fault lies under
> On Thu, Oct 18, 2012 at 4:00 PM, Krishna Kishore Bonagiri
> <[EMAIL PROTECTED]> wrote:
> > Hi,
> > Is there a way we can ask the YARN RM for not killing a container when
> > uses excess virtual memory than the maximum it can use as per the
> > specification in the configuration file yarn-site.xml? We can't always
> > estimate the amount of virtual memory needed for our application running
> > a container, but we don't want to get it killed in a case it exceeds the
> > maximum limit.
> > Please suggest as to how can we come across this issue.
> > Thanks,
> > Kishore
> Harsh J