Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> RE: LocalResource size/time limits


Copy link to this message
-
RE: LocalResource size/time limits
Fantastic.  Just what I needed to know.
john

From: Hardik Pandya [mailto:[EMAIL PROTECTED]]
Sent: Saturday, January 04, 2014 11:15 AM
To: [EMAIL PROTECTED]
Subject: Re: LocalResource size/time limits

May be this would clarify some aspect of your questions
Resource Localization in YARN Deep Dive<http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/>

The threshold for local files is dictated by the configuration property yarn.nodemanager.localizer.cache.target-size-mb described below.
http://hadoop.apache.org/docs/current/hadoop-yarn/hadoop-yarn-common/yarn-default.xml

yarn.nodemanager.localizer.cache.target-size-mb

10240

Target size of localizer cache in MB, per local directory.
On Sat, Jan 4, 2014 at 12:02 PM, John Lilley <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Are there any limits on the total size of LocalResources that a YARN app requests?  Do the PUBLIC ones age out of cache over time?  Are there settable controls?
Thanks
John

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB