Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Data in File-channel  data folder


+
Madhu Gmail 2013-04-11, 16:52
Copy link to this message
-
Re: Data in File-channel  data folder
Madhu Gmail 2013-04-11, 19:13
Hello,

I have not heard from anyone.  so just want make sure I have explained the issue correctly.

I think this is a common problem for everyone who uses it flume.

when flume sink consumes the log event from file channel,  what will happen to the data that is committed to local disk under data directory.

will it grow indefinitely  like log-1, log-2, log-3.....and so on ???

do I have to write script to remove the data from data directory  ??

Madhu  Munagala
(214)679-2872

On Apr 11, 2013, at 11:52 AM, Madhu Gmail <[EMAIL PROTECTED]> wrote:

> Hello,
>
> How to clean up the data  in file channel data folder.  After the log events are processed by the sink,  I still see the log-1 and log-2 shows 1.6GB and 1.2GB.
>
> once the log events are processed by the sink,  the channel should not have any data in data directory under file-channel ....??
>
>
> Madhu  Munagala
> (214)679-2872
+
Hari Shreedharan 2013-04-11, 19:57
+
Madhu Gmail 2013-04-11, 20:11
+
Mike Keane 2013-04-11, 20:08
+
Madhu Gmail 2013-04-11, 20:17