Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Flume Sink Cleanup


+
Rajesh Jain 2013-06-20, 14:59
Copy link to this message
-
Re: Flume Sink Cleanup
Hi Rajesh,

You have to write a cron job, a overwrite after some time isn't possible.

Regards,
 Alex

On Jun 20, 2013, at 4:59 PM, Rajesh Jain <[EMAIL PROTECTED]> wrote:

> Hi
>
> This is a newbie question.
>
> I have configured flume sink with the Rolling File Sink
>
> I have the following in my flume-conf.properties page
>
> agent1.sinks.purepath.type = com.x.diagnostics.flume.RollingFileSink
> # once an hour
> agent1.sinks.purepath.sink.rollInterval = 3600
> # Force cutoff at 100 MB
> agent1.sinks.purepath.sink.rollSize = 100
> agent1.sinks.purepath.sink.directory = /log/export
> agent1.sinks.purepath.batchSize = 1000
>
> The directory fills up and is there a way to clean it up. Do I have to write an external cron job to clean up the directory, or is there a way to overwrite the files (recycle) after a specified period - say 6 months
>
> Thanks, Rajesh
>

--
Alexander Alten-Lorenz
http://mapredit.blogspot.com
German Hadoop LinkedIn Group: http://goo.gl/N8pCF