Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> reuse cached files


Copy link to this message
-
Re: reuse cached files
Hi,

> if I use distributed cache to send some files to all the nodes in one MR job,
> can I reuse these cached files locally in my next job, or will hadoop re-sent
> these files again?

Cache files are reused across Jobs. From trunk onwards, they will be
restricted to be reused across jobs of the same user, unless they are
marked 'public' in which case they can be reused by jobs across all
users.

Thanks
hemanth
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB