Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> A backup sink

Copy link to this message
Re: A backup sink
You would want to replicate events to two channels and have something
like a file sink pull from the second channel. You may want to look at

As to using headers to determine path, you use the hdfs sink but point
it to the local filesystem. The HDFS sink allows header substitution in
the path.

On 08/11/2013 04:49 PM, Anat Rozenzon wrote:
> Hi,
> I'm looking for a way to create a sink that will simply 'backup' the
> files I'm sending through an Avro Source.
> I have in the headers the file name, host name etc.
> Is there a way to do store each line in a file according to these headers?
> For example store events in a file called
> orig_hostname.orig_filename.timestamp, e.g.:
> myhost.messages.log.1.135678263178
> Thanks
> Anat