Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # dev - Review Request 13463: FLUME-2157. Spool directory source does not shut down correctly when Flume is reconfigured.


Copy link to this message
-
Review Request 13463: FLUME-2157. Spool directory source does not shut down correctly when Flume is reconfigured.
Mike Percy 2013-08-10, 02:33

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13463/
-----------------------------------------------------------

Review request for Flume.
Bugs: FLUME-2157
    https://issues.apache.org/jira/browse/FLUME-2157
Repository: flume-git
Description
-------

Spool directory source does not shut down correctly when Flume is reconfigured. This can lead to all sorts of problems stemming from the fact that we have threads running which should not be. In the logs, it looks like multiple spool directory sources are running on the same directory, even though the config may say otherwise.
Diffs
-----

  flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java f82fe1f
  flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 7145580
  flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java 652d2a2
  flume-ng-tests/src/test/java/org/apache/flume/test/agent/TestSpooldirSource.java PRE-CREATION
  flume-ng-tests/src/test/java/org/apache/flume/test/util/StagedInstall.java bc58340

Diff: https://reviews.apache.org/r/13463/diff/
Testing
-------

Added a unit test for this specific case and also an integration test that simply runs an agent with many concurrent spool directory sources.
Thanks,

Mike Percy