Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - reuse cached files


Copy link to this message
-
Re: reuse cached files
Hemanth Yamijala 2010-07-30, 04:24
Hi,

> if I use distributed cache to send some files to all the nodes in one MR job,
> can I reuse these cached files locally in my next job, or will hadoop re-sent
> these files again?

Cache files are reused across Jobs. From trunk onwards, they will be
restricted to be reused across jobs of the same user, unless they are
marked 'public' in which case they can be reused by jobs across all
users.

Thanks
hemanth