Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
MapReduce >> mail # dev >> How not to clean MapReduce temp data?


Copy link to this message
-
Re: How not to clean MapReduce temp data?
Adding to Bejoy's response, if you want to just keep the temp output
of failed tasks, you can also use keep.failed.task.files.

Also, these config variable names are as per Hadoop 1.0. In trunk,
they are called mapreduce.task.files.preserve.filepattern and
mapreduce.task.files.preserve.failedtasks respectively.

Thanks
Hemanth

On Thu, Sep 27, 2012 at 11:18 PM, Bejoy Ks <[EMAIL PROTECTED]> wrote:
> Hi
>
> The temporary output from tasks can be preserved using the following
> property'keep.task.files.pattern'
>
> http://books.google.co.in/books?id=drbI_aro20oC&pg=PA178&lpg=PA178&dq=keep.task.files.pattern&source=bl&ots=tZAmxgm_j4&sig=Guc0bh2BQzlbMqOADtic5WciIz0&hl=en&sa=X&ei=zI9kULbDM8zhrAe3jYH4BA&ved=0CFoQ6AEwBg#v=onepage&q=keep.task.files.pattern&f=false
>
>
> On Thu, Sep 27, 2012 at 10:49 PM, Pedro Sá da Costa <[EMAIL PROTECTED]>
> wrote:
>>
>> Hi,
>>
>> Is there a parameter to avoid to clean the temporary data created during
>> the job execution after the job end?
>>
>> --
>> Best regards,
>>
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB