Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - RE: LocalResource size/time limits


Copy link to this message
-
RE: LocalResource size/time limits
John Lilley 2014-01-05, 21:23
Fantastic.  Just what I needed to know.
john

From: Hardik Pandya [mailto:[EMAIL PROTECTED]]
Sent: Saturday, January 04, 2014 11:15 AM
To: [EMAIL PROTECTED]
Subject: Re: LocalResource size/time limits

May be this would clarify some aspect of your questions
Resource Localization in YARN Deep Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>

The threshold for local files is dictated by the configuration property yarn.nodemanager.localizer.cache.target-size-mb described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

yarn.nodemanager.localizer.cache.target-size-mb

10240

Target size of localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Are there any limits on the total size of LocalResources that a YARN app requests?  Do the PUBLIC ones age out of cache over time?  Are there settable controls?
Thanks
John