Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request: FLUME-1947. ExecSource should optionally block if the channel is full.


Copy link to this message
-
Review Request: FLUME-1947. ExecSource should optionally block if the channel is full.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9914/
-----------------------------------------------------------

Review request for Flume.
Description
-------

Made multiple changes to the exec source:

1. Flush the events to the channel, based on a configurable timeout - even when a full batch was not received.
2. Block the spawned process until channel exceptions are cleared up by not reading the stdout buffer. Backoff exponentially (subject to an upper limit), till events are available.
This addresses bug FLUME-1947.
    https://issues.apache.org/jira/browse/FLUME-1947
Diffs
-----

  flume-ng-core/src/main/java/org/apache/flume/source/ExecSource.java 8e687f2
  flume-ng-core/src/main/java/org/apache/flume/source/ExecSourceConfigurationConstants.java fd5a60b
  flume-ng-core/src/test/java/org/apache/flume/source/TestExecSource.java 7c573f6

Diff: https://reviews.apache.org/r/9914/diff/
Testing
-------

Added unit tests for the new features. All current tests pass.
Thanks,

Hari Shreedharan