Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop >> mail # user >> how to get rid of attempt_201101170925_****_m_**** directories safely?


Copy link to this message
-
Re: how to get rid of attempt_201101170925_****_m_**** directories safely?
On Thu, Mar 17, 2011 at 1:20 PM, jigar shah <[EMAIL PROTECTED]> wrote:
> Hi,
>
>    we are running a 50 node hadoop cluster and have a problem with these
> attempt directories piling up(for eg attempt_201101170925_126956_m_000232_0)
> and taking a lot of space. when i restart the tasktracker daemon these
> directories get cleaned out and the space usage goes down.
>
>    i understand why these directories are created, but it becomes a pain
> when they just hang around indefinitely. its very inconvenient to restart
> the tasktracker to get rid of them  and reclaim space.
>
>    anyone knows if there is a setting in the conf somewhere i can set that
> will periodically prune these directories or any other way to deal with
> this.
>
>    i appreciate any sort of help
>
>
>
> thanks
>
>
>

Something you can run from cron.
6 3 * * * hadoop find /disk6/hadoop_root/hdfs_data/hadoop/mapred/local
-maxdepth 1 -name "attempt_*" -ctime +7 -delete
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB