Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
HDFS >> mail # user >> Re: Auto clean DistCache?


Copy link to this message
-
Re: Auto clean DistCache?

You can control the limit of these cache files, the default is 10GB (value of 10737418240L): Try changing local.cache.size or mapreduce.tasktracker.cache.local.size in mapred-site.xml

Thanks,
+Vinod Kumar Vavilapalli
Hortonworks Inc.
http://hortonworks.com/

On Mar 25, 2013, at 5:16 PM, Jean-Marc Spaggiari wrote:

> Hi,
>
> Each time my MR job is run, a directory is created on the TaskTracker
> under mapred/local/taskTracker/hadoop/distcache (based on my
> configuration).
>
> I looked at the directory today, and it's hosting thousands of
> directories and more than 8GB of data there.
>
> Is there a way to automatically delete this directory when the job is done?
>
> Thanks,
>
> JM

NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB