Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> How to control synchronization on spooldir

Copy link to this message
How to control synchronization on spooldir

I have a rolling file appender which writes log files to a spooldirectory,
and an agent which polls this directory and copies the data to a log server.

1.  How do I control read/write?
2.  How do I use the the generated events of fileHeader and fileHeaderKey?

My source is a "spooldir" and "sink" is avro sink?

On the target side.

source is "avro" and target is "file".

I am trying to do is poll a source directory and copy them onto the target
directory, I would like to keep the filenames.

Can anyone provide some sample configuration files, how this "fileheader"
and "fileheaderkey" can be used, that would be great.