Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Review Request 12211: FLUME-1899 - Make the spool directory source work with sub directories


+
Phil Scala 2013-07-01, 15:53
+
Alexander Alten-Lorenz 2013-07-17, 17:05
Copy link to this message
-
Re: Review Request 12211: FLUME-1899 - Make the spool directory source work with sub directories


> On July 17, 2013, 5:05 p.m., Alexander Alten-Lorenz wrote:
> > Can you please reformat to remove the leading spaces and empty lines? They're marked red when you view the diff.
> > Would you create a patch for our docs too?

Hi Alexender, I can it will take me some time, as I am busy on a business trip and do not have the environment 100% setup with me.  If you choose "hide whitespace changes" in the review board diff, then it looks pretty reasonable to me.   Let me know if that's not sufficient.

And, yes, the patch included the user guide changes.
- Phil
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/12211/#review23295
-----------------------------------------------------------
On July 1, 2013, 3:53 p.m., Phil Scala wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/12211/
> -----------------------------------------------------------
>
> (Updated July 1, 2013, 3:53 p.m.)
>
>
> Review request for Flume.
>
>
> Bugs: https://issues.apache.org/jira/browse/FLUME-1899
>     https://issues.apache.org/jira/browse/https://issues.apache.org/jira/browse/FLUME-1899
>
>
> Repository: flume-git
>
>
> Description
> -------
>
> Added recursive directory walking to the ReliableSpoolingFileEventReader.  the same ignore pattern is used against the sub-directory names as well.
>
>
> Diffs
> -----
>
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java f82fe1f
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 641b5c6
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java f3cc703
>   flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java 652d2a2
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 1b4d216
>
> Diff: https://reviews.apache.org/r/12211/diff/
>
>
> Testing
> -------
>
> Added new unit test, local testing on windows and CentOS.  By default the setting is not enabled, need to update configuration i.e. spool-agent.sources.spooler.recursiveDirectorySearch = true
>
>
> Thanks,
>
> Phil Scala
>
>

+
Alexander Alten-Lorenz 2013-07-17, 19:27
+
Phil Scala 2013-07-25, 18:31
+
Phil Scala 2013-07-25, 18:40