Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> How to control synchronization on spooldir


Copy link to this message
-
How to control synchronization on spooldir
Hi,

I have a rolling file appender which writes log files to a spooldirectory,
and an agent which polls this directory and copies the data to a log server.

1.  How do I control read/write?
2.  How do I use the the generated events of fileHeader and fileHeaderKey?

My source is a "spooldir" and "sink" is avro sink?

On the target side.

source is "avro" and target is "file".

I am trying to do is poll a source directory and copy them onto the target
directory, I would like to keep the filenames.

Can anyone provide some sample configuration files, how this "fileheader"
and "fileheaderkey" can be used, that would be great.

Thanks,
yogi
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB