Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Hadoop, mail # user - how to get rid of attempt_201101170925_****_m_**** directories safely?


Copy link to this message
-
Re: how to get rid of attempt_201101170925_****_m_**** directories safely?
Edward Capriolo 2011-03-18, 02:15
On Thu, Mar 17, 2011 at 1:20 PM, jigar shah <[EMAIL PROTECTED]> wrote:
> Hi,
>
>    we are running a 50 node hadoop cluster and have a problem with these
> attempt directories piling up(for eg attempt_201101170925_126956_m_000232_0)
> and taking a lot of space. when i restart the tasktracker daemon these
> directories get cleaned out and the space usage goes down.
>
>    i understand why these directories are created, but it becomes a pain
> when they just hang around indefinitely. its very inconvenient to restart
> the tasktracker to get rid of them  and reclaim space.
>
>    anyone knows if there is a setting in the conf somewhere i can set that
> will periodically prune these directories or any other way to deal with
> this.
>
>    i appreciate any sort of help
>
>
>
> thanks
>
>
>

Something you can run from cron.
6 3 * * * hadoop find /disk6/hadoop_root/hdfs_data/hadoop/mapred/local
-maxdepth 1 -name "attempt_*" -ctime +7 -delete