Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Wildcard path for spooling directory source?


Copy link to this message
-
RE: Wildcard path for spooling directory source?
Phil Scala 2013-06-27, 17:35
Nathan, the current code base does not have subdirectory support,

If it's any consolation , I have a working patch to the Flume 1.4 code base for an enhancement to add support for sub directories (FLUME-1899<https://issues.apache.org/jira/browse/FLUME-1899>).    I just need to dedicate some time to finalize testing.  For now you would have to keep all of the files in the same directory.

Thanks
Phil
Phil Scala
Software Developer / Architect
Global Relay

[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>

866.484.6630  |  [EMAIL PROTECTED]<https://ex1.globalrelay.net/owa/redir.aspx?C=01df0579a4cc42feb9f7e2801368f729&URL=mailto%3ainfo%40globalrelay.net>  |  globalrelay.com<https://ex1.globalrelay.net/owa/redir.aspx?C=01df0579a4cc42feb9f7e2801368f729&URL=http%3a%2f%2fwww.globalrelay.com%2f>

From: Nathan Grice [mailto:[EMAIL PROTECTED]]
Sent: Thursday, June 27, 2013 1:31 PM
To: [EMAIL PROTECTED]
Subject: Wildcard path for spooling directory source?

Does the spooling directory source support wilcards in the path?
I am looking to watch a list of directories without having to reconfigure the agent when a new path is added.
Example:

spoolingDir = /var/log/sources/*

and when I add a new directory to the /var/log/sources dir, such as  /var/log/sources/1,  /var/log/sources/2, etc, the files placed there are automatically picked up by the agent.

Thanks