Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Review Request 12211: FLUME-1899 - Make the spool directory source work with sub directories


Copy link to this message
-
Review Request 12211: FLUME-1899 - Make the spool directory source work with sub directories

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12211/
-----------------------------------------------------------

Review request for Flume.
Bugs: https://issues.apache.org/jira/browse/FLUME-1899
    https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1899
Repository: flume-git
Description
-------

Added recursive directory walking to the ReliableSpoolingFileEventReader.  the same ignore pattern is used against the sub-directory names as well.
Diffs
-----

  flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java f82fe1f
  flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 641b5c6
  flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java f3cc703
  flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java 652d2a2
  flume-ng-doc/sphinx/FlumeUserGuide.rst 1b4d216

Diff: https://reviews.apache.org/r/12211/diff/
Testing
-------

Added new unit test, local testing on windows and CentOS.  By default the setting is not enabled, need to update configuration i.e. spool-agent.sources.spooler.recursiveDirectorySearch = true
Thanks,

Phil Scala

+
Alexander Alten-Lorenz 2013-07-17, 17:05
+
Phil Scala 2013-07-17, 19:04
+
Alexander Alten-Lorenz 2013-07-17, 19:27
+
Phil Scala 2013-07-25, 18:31
+
Phil Scala 2013-07-25, 18:40