Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Review Request: ExecSource don't flush the buffer if there are no input events and the batch size has not reached


Copy link to this message
-
Review Request: ExecSource don't flush the buffer if there are no input events and the batch size has not reached

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10632/
-----------------------------------------------------------

Review request for Flume.
Description
-------

The Fix is to add "batchTimeout" Property for ExecSource (default is 3 seconds). if batchTimeout exceeded, ExecSource will push all buffered data to the channel even is the batch size was not reached.

Pls. see the Jira issue (https://issues.apache.org/jira/browse/FLUME-1819) for more details.
This addresses bug FLUME-1819.
    https://issues.apache.org/jira/browse/FLUME-1819
Diffs
-----
Diff: https://reviews.apache.org/r/10632/diff/
Testing
-------

1. Added a Unit Test case to the class TestExecSource class.
2. Tested with a flume agent locally. I setup source as a "tail -f <file>" command, used as in-memory channel and the Sink as "logger".
Thanks,

Venkatesh Sivasubramanian

+
Jeff jlord 2013-04-19, 20:54