Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: issue about the MR JOB local dir


Copy link to this message
-
Re: issue about the MR JOB local dir
thank you ,but it seems the doc is littler old , doc says
   - *PUBLIC:* <local-dir>/filecache
   - *PRIVATE:* <local-dir>/usercache//filecache
   - *APPLICATION:* <local-dir>/usercache//appcache/<app-id>/

but here is my nodemanager directory,i guess nmPrivate belongs to private
dir ,and filecache dir is not exist in usercache

# ls /data/mrlocal/1/yarn/local
filecache  nmPrivate  usercache
[root@CHBM223 conf]# ls /data/mrlocal/1/yarn/local/filecache/
-1058429088916409723  4529188628984375230   -7014620238384418063
1084965746802723478   4537624275313838973   -7168597014714301440
-1624997938511480096  4630901056913375526   7270199361370573766
-1664837184667315424  -4642830643595652223  -7332220817185869511
1725675017861848111   471523682744090087    7332904188082338506
1838346487029342338   4790459366530674957   -7450121760156930096
186504478230003977    4800525395984004560   7478948409771297223
-2348110367263014791  -5080956154405911478  7486468764131639983
-2569725565520513438  524923119076958393    -7755253483162230956
-2590767617048813033  -5270961733852362332  -7859425335924192987
2787947055181616358   -5381775829268220744  7967711417630616031
2816094634154232444   -5845090920164902899  8115657316961272063
286373945366133510    -587409153437667574   -8196745140008584754
2931191327895309259   -5951079387471670627  -8338714062663466224
-304471400571947298   -6076923167039033115  -8473967805299855837
325019546688058584    6080416638029534254   8513492322348652110
-3331048722364108374  6332597539903254606   -8567312237113801580
3360339691049457808   6343222208406792699   8737308241488535006
336835441200377451    6566344665060319340   -8893869581665287805
3628504729266619560   -6639258108397695527  -8895898681278542021
-3801380133229678986  -6653760362065293300  8926294383627727352
3837066533086156807   -6782198269120858036  -8964326004503603190
3929223016635331138   -6814427383139267223  -9049325747073392755
4126862917222506438   -6814979781017122863  -9186700026428986961
[root@CHBM223 conf]# ls /data/mrlocal/1/yarn/local/nmPrivate/
application_1385444985453_0001  application_1385453784842_0010
application_1385522685434_0081
container_1385522685434_0073_01_000014.pid
application_1385445543402_0003  application_1385453784842_0013
container_1385522685434_0073_01_000005.pid
container_1385522685434_0073_01_000017.pid
application_1385445543402_0005  application_1385520079773_0005
container_1385522685434_0073_01_000008.pid
[root@CHBM223 conf]# ls /data/mrlocal/1/yarn/local/usercache/
hdfs  helen  hive  root

On Thu, Dec 5, 2013 at 5:12 AM, Jian He <[EMAIL PROTECTED]> wrote:

> The following links may help you
> http://hortonworks.com/blog/management-of-application-dependencies-in-yarn/
> http://hortonworks.com/blog/resource-localization-in-yarn-deep-dive/
>
> Thanks,
> Jian
>
>
> On Tue, Dec 3, 2013 at 5:26 PM, ch huang <[EMAIL PROTECTED]> wrote:
>
>> hi,maillist:
>>             i see three dirs on my local MR job dir ,and i do not know
>> these dirs usage,anyone knows?
>>
>> # ls /data/1/mrlocal/yarn/local/
>> filecache/ nmPrivate/ usercache/
>>
>
>
> CONFIDENTIALITY NOTICE
> NOTICE: This message is intended for the use of the individual or entity
> to which it is addressed and may contain information that is confidential,
> privileged and exempt from disclosure under applicable law. If the reader
> of this message is not the intended recipient, you are hereby notified that
> any printing, copying, dissemination, distribution, disclosure or
> forwarding of this communication is strictly prohibited. If you have
> received this communication in error, please contact the sender immediately
> and delete it from your system. Thank You.