Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: issue about the MR JOB local dir


Copy link to this message
-
Re: issue about the MR JOB local dir
thank you ,but it seems the doc is littler old , doc says
   - *PUBLIC:* <local-dir>/filecache
   - *PRIVATE:* <local-dir>/usercache//filecache
   - *APPLICATION:* <local-dir>/usercache//appcache/<app-id>/

but here is my nodemanager directory,i guess nmPrivate belongs to private
dir ,and filecache dir is not exist in usercache

# ls /data/mrlocal/1/yarn/local
filecache  nmPrivate  usercache
[root@CHBM223 conf]# ls /data/mrlocal/1/yarn/local/filecache/
-1058429088916409723  4529188628984375230   -7014620238384418063
1084965746802723478   4537624275313838973   -7168597014714301440
-1624997938511480096  4630901056913375526   7270199361370573766
-1664837184667315424  -4642830643595652223  -7332220817185869511
1725675017861848111   471523682744090087    7332904188082338506
1838346487029342338   4790459366530674957   -7450121760156930096
186504478230003977    4800525395984004560   7478948409771297223
-2348110367263014791  -5080956154405911478  7486468764131639983
-2569725565520513438  524923119076958393    -7755253483162230956
-2590767617048813033  -5270961733852362332  -7859425335924192987
2787947055181616358   -5381775829268220744  7967711417630616031
2816094634154232444   -5845090920164902899  8115657316961272063
286373945366133510    -587409153437667574   -8196745140008584754
2931191327895309259   -5951079387471670627  -8338714062663466224
-304471400571947298   -6076923167039033115  -8473967805299855837
325019546688058584    6080416638029534254   8513492322348652110
-3331048722364108374  6332597539903254606   -8567312237113801580
3360339691049457808   6343222208406792699   8737308241488535006
336835441200377451    6566344665060319340   -8893869581665287805
3628504729266619560   -6639258108397695527  -8895898681278542021
-3801380133229678986  -6653760362065293300  8926294383627727352
3837066533086156807   -6782198269120858036  -8964326004503603190
3929223016635331138   -6814427383139267223  -9049325747073392755
4126862917222506438   -6814979781017122863  -9186700026428986961
[root@CHBM223 conf]# ls /data/mrlocal/1/yarn/local/nmPrivate/
application_1385444985453_0001  application_1385453784842_0010
application_1385522685434_0081
container_1385522685434_0073_01_000014.pid
application_1385445543402_0003  application_1385453784842_0013
container_1385522685434_0073_01_000005.pid
container_1385522685434_0073_01_000017.pid
application_1385445543402_0005  application_1385520079773_0005
container_1385522685434_0073_01_000008.pid
[root@CHBM223 conf]# ls /data/mrlocal/1/yarn/local/usercache/
hdfs  helen  hive  root

On Thu, Dec 5, 2013 at 5:12 AM, Jian He <[EMAIL PROTECTED]> wrote:

> The following links may help you
> http://hortonworks.com/blog/management-of-application-dependencies-in-yarn/
> http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/
>
> Thanks,
> Jian
>
>
> On Tue, Dec 3, 2013 at 5:26 PM, ch huang <[EMAIL PROTECTED]> wrote:
>
>> hi,maillist:
>>             i see three dirs on my local MR job dir ,and i do not know
>> these dirs usage,anyone knows?
>>
>> # ls /data/1/mrlocal/yarn/local/
>> filecache/ nmPrivate/ usercache/
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB