Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request: ExecSource don't flush the buffer if there are no input events and the batch size has not reached


Copy link to this message
-
Re: Review Request: ExecSource don't flush the buffer if there are no input events and the batch size has not reached

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10632/#review19476
-----------------------------------------------------------

Ship it!
+1

- Jeff jlord
On April 19, 2013, 7:02 a.m., Venkatesh Sivasubramanian wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10632/
> -----------------------------------------------------------
>
> (Updated April 19, 2013, 7:02 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> The Fix is to add "batchTimeout" Property for ExecSource (default is 3 seconds). if batchTimeout exceeded, ExecSource will push all buffered data to the channel even is the batch size was not reached.
>
> Pls. see the Jira issue (https://issues.apache.org/jira/browse/FLUME-1819) for more details.
>
>
> This addresses bug FLUME-1819.
>     https://issues.apache.org/jira/browse/FLUME-1819
>
>
> Diffs
> -----
>
>
> Diff: https://reviews.apache.org/r/10632/diff/
>
>
> Testing
> -------
>
> 1. Added a Unit Test case to the class TestExecSource class.
> 2. Tested with a flume agent locally. I setup source as a "tail -f <file>" command, used as in-memory channel and the Sink as "logger".
>
>
> Thanks,
>
> Venkatesh Sivasubramanian
>
>