-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6377/#review12907
-----------------------------------------------------------
Sorry for the delay in this review, I was out of town last week. There is still an issue with the BufferedReader / mark / reset semantics, please see below for details.
flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java
<https://reviews.apache.org/r/6377/#comment27742>

    Please add an note to the javadocs saying that the SpoolingFileLineReader class is ONLY for internal use by Flume components, not for developers extending Flume.

flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java
<https://reviews.apache.org/r/6377/#comment27745>

    If the line length exceeds the length of the BufferedReader, this reset() call will throw a java.io.IOException: Mark invalid
    
    Also, committed = true when you hit this case, since committed = false does not get set until right before the return statement.
    
    Therefore, assuming the client catches the exception and recovers from it, the next call to readLines(n) will simply skip over those lines and the data will be lost.
    
    So if there is an error in which the line is too long, since we have decided that we going to fail then we should fail explicitly and permanently:
    1. Throw a FlumeException indicating a too-long line was seen (do not try to reset the reader) and also:
    2. Permanently disable the SpoolingFileLineReader object so that the next time someone tries to call any operation on it, such as readLines(), commit(), or close(), throw an IllegalStateException indicating that it is no longer in a usable state.
    
    In a future revision of this implementation, we can do something a little more graceful, but we have to ensure that we don't lose data in any case.

flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java
<https://reviews.apache.org/r/6377/#comment27740>

    This should be something like:
    
    int bufferSize = bufferMaxLines * bufferMaxLineLength;
    BufferedReader reader = new BufferedReader(new FileReader(nextFile), bufferSize);
    reader.mark(bufferSize);
    
    ...since the default buffer size in the BufferedReader constructor is 8124 in JDK 6 and that may not be enough to hold your bufferSize (or it may be too much). Also, the edge-case behavior of BufferedReader (w.r.t. mark and reset) is different in situations where the constructor arg is different than the arg passed to mark(), so rather than deal with that inconsistency it makes sense to do this.

flume-ng-core/src/test/java/org/apache/flume/client/avro/TestSpoolingFileLineReader.java
<https://reviews.apache.org/r/6377/#comment27746>

    This test is failing for me. Not sure why, haven't dug into it much yet.

flume-ng-core/src/test/java/org/apache/flume/client/avro/TestSpoolingFileLineReader.java
<https://reviews.apache.org/r/6377/#comment27747>

    It's failing on this assert

flume-ng-core/src/test/java/org/apache/flume/client/avro/TestSpoolingFileLineReader.java
<https://reviews.apache.org/r/6377/#comment27741>

    This line is not long enough to exceed the buffer size in the mark() call, so we are not testing the case of an extremely long line. We need to test the situation where reset() throws an exception, since we will lose data in that case with the current implementation.
    
    While a barely-too-long line is a valid case to test, we also need a case that tests the extremely long line scenario, i.e.:
    
    // make a prefix string as long as the internal buffer size
    StringBuilder sb = new StringBuilder();
    for (int i = 0; i < maxLines * maxLineLength; i++) {
      sb.append('x');
    }
    String lotsofXs = sb.toString();
    
    Files.write("file1line1\nfile1line2\nfile1line3\nfile1line4\n" +
      "file1line5\nfile1line6\nfile1line7\nfile1line8\n" +
      lotsOfXs + " reallyreallyreallyreallyLongfile1line9\n" +  // <-- line exceeds BufferedReader internal buf
      "file1line10\nfile1line11\nfile1line12\nfile1line13\n",
      f1, Charsets.UTF_8);
- Mike Percy
On Oct. 22, 2012, 8:36 p.m., Patrick Wendell wrote:

> On Oct. 30, 2012, 6:22 a.m., Mike Percy wrote:
> > flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java, line 300
> > <https://reviews.apache.org/r/6377/diff/5/?file=178797#file178797line300>
> >
> >     This should be something like:
> >    
> >     int bufferSize = bufferMaxLines * bufferMaxLineLength;
> >     BufferedReader reader = new BufferedReader(new FileReader(nextFile), bufferSize);
> >     reader.mark(bufferSize);
> >    
> >     ...since the default buffer size in the BufferedReader constructor is 8124 in JDK 6 and that may not be enough to hold your bufferSize (or it may be too much). Also, the edge-case behavior of BufferedReader (w.r.t. mark and reset) is different in situations where the constructor arg is different than the arg passed to mark(), so rather than deal with that inconsistency it makes sense to do this.

Ya good catch.
> On Oct. 30, 2012, 6:22 a.m., Mike Percy wrote:
> > flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java, line 188
> > <https://reviews.apache.org/r/6377/diff/5/?file=178797#file178797line188>
> >
> >     If the line length exceeds the length of the BufferedReader, this reset() call will throw a java.io.IOException: Mark invalid
> >    
> >     Also, committed = true when you hit this case, since committed = false does not get set until right before the return statement.
> >    
> >     Therefore, assuming the client catches the exception and recovers from it, the next call to readLines(n) will simply skip over those lines and the data will be lost.
> >    
> >     So if there is an error in which the line is too long, since we have decided that we going to fail then we should fail explicitly and permanently:
> >     1. Throw a FlumeException indicating a too-long line was seen (do not try to reset the reader) and also:
> >     2. Permanently disable the SpoolingFileLineReader object so that the next time someone tries to call any operation on it, such as readLines(), commit(), or close(), throw an IllegalStateException indicating that it is no longer in a usable state.
> >    
> >     In a future revision of this implementation, we can do something a little more graceful, but we have to ensure that we don't lose data in any case.

Yep this should stop the world when it fails... my mistake. Added some tests to make sure this works.
> On Oct. 30, 2012, 6:22 a.m., Mike Percy wrote:
> > flume-ng-core/src/test/java/org/apache/flume/client/avro/TestSpoolingFileLineReader.java, line 363
> > <https://reviews.apache.org/r/6377/diff/5/?file=178801#file178801line363>
> >
> >     This test is failing for me. Not sure why, haven't dug into it much yet.

I can't yet reproduce this failure.
- Patrick
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6377/#review12907
-----------------------------------------------------------
On Oct. 22, 2012, 8:36 p.m., Patrick Wendell wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6377/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2012, 8:36 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This patch adds a spooling directory based source. The  idea is that a user can have a spool directory where files are deposited for ingestion into flume. Once ingested, the files are clearly renamed and the implementation guarantees at-least-once delivery semantics similar to those achieved within flume itself, even across failures and restarts of the JVM running the code.
>
> This helps fill the gap for people who want a way to get reliable delivery of events into flume, but don't want to directly write their application against the flume API. They can simply drop log files off in a spooldir and let flume ingest asynchronously (using some shell scripts or other automated process).

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6377/#review13120
-----------------------------------------------------------
Hi Patrick, overall this looks good to me. Only thing is the test failure. I am tempted to just commit this, but I'm trying to understand what's going on with the test.

It looks to me that the test is trying to assert that the files not be processed if they are empty. I added some debug messages to the assert and this is what it is printing:

testBehaviorWithEmptyFile(org.apache.flume.client.avro.TestSpoolingFileLineReader)  Time elapsed: 1.012 sec  <<< FAILURE!
java.lang.AssertionError: Does not contain file: /var/folders/fg/ym7gqsvs5h30gg13yjpt1hm00000gn/T/1352164166517-0/file1, only contains: [/var/folders/fg/ym7gqsvs5h30gg13yjpt1hm00000gn/T/1352164166517-0/file1.COMPLETE, /var/folders/fg/ym7gqsvs5h30gg13yjpt1hm00000gn/T/1352164166517-0/file2.COMPLETE]
  at org.junit.Assert.fail(Assert.java:93)
  at org.junit.Assert.assertTrue(Assert.java:43)
  at org.apache.flume.client.avro.TestSpoolingFileLineReader.testBehaviorWithEmptyFile(TestSpoolingFileLineReader.java:399)

So the test is asserting that it should not have rolled test1 to test1.COMPLETE, but it has (At the end of this test, it has actually run both).

I don't see any code in the patch indicating that files that have no data should not be read. So I don't understand how this test could be passing for you.

Please try re-running the test using this command from the Flume top-level: mvn clean install -Dtest=TestSpoolingFileLineReader -DfailIfNoTests=false ... I will be very surprised if it passes.

I am still reviewing this patch and am currently leaning toward adding an @Ignore to this test for commit, since I'd like to get this feature into 1.3.0. In the mean time, any comments on the desired behavior are appreciated.

Regards,
Mike
- Mike Percy
On Oct. 22, 2012, 8:36 p.m., Patrick Wendell wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6377/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2012, 8:36 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This patch adds a spooling directory based source. The  idea is that a user can have a spool directory where files are deposited for ingestion into flume. Once ingested, the files are clearly renamed and the implementation guarantees at-least-once delivery semantics similar to those achieved within flume itself, even across failures and restarts of the JVM running the code.
>
> This helps fill the gap for people who want a way to get reliable delivery of events into flume, but don't want to directly write their application against the flume API. They can simply drop log files off in a spooldir and let flume ingest asynchronously (using some shell scripts or other automated process).
>
> Unlike the prior iteration, this patch implements a first-class source. It also extends the avro client to support spooling in a similar manner.
>
>
> This addresses bug FlUME-1425.
>     https://issues.apache.org/jira/browse/FlUME-1425
>
>
> Diffs
> -----
>
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceConfiguration.java da804d7
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java abbbf1c
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/AvroCLIClient.java 4a5ecae
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/BufferedLineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/LineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java PRE-CREATION

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6377/#review13121
-----------------------------------------------------------

Ship it!
Hey Patrick, really good work overall. I'm going to commit this as-is. We can follow-up more after the initial commit.

- Mike Percy
On Oct. 22, 2012, 8:36 p.m., Patrick Wendell wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6377/
> -----------------------------------------------------------
>
> (Updated Oct. 22, 2012, 8:36 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This patch adds a spooling directory based source. The  idea is that a user can have a spool directory where files are deposited for ingestion into flume. Once ingested, the files are clearly renamed and the implementation guarantees at-least-once delivery semantics similar to those achieved within flume itself, even across failures and restarts of the JVM running the code.
>
> This helps fill the gap for people who want a way to get reliable delivery of events into flume, but don't want to directly write their application against the flume API. They can simply drop log files off in a spooldir and let flume ingest asynchronously (using some shell scripts or other automated process).
>
> Unlike the prior iteration, this patch implements a first-class source. It also extends the avro client to support spooling in a similar manner.
>
>
> This addresses bug FlUME-1425.
>     https://issues.apache.org/jira/browse/FlUME-1425
>
>
> Diffs
> -----
>
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceConfiguration.java da804d7
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java abbbf1c
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/AvroCLIClient.java 4a5ecae
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/BufferedLineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/LineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java PRE-CREATION
>   flume-ng-core/src/test/java/org/apache/flume/client/avro/TestBufferedLineReader.java PRE-CREATION
>   flume-ng-core/src/test/java/org/apache/flume/client/avro/TestSpoolingFileLineReader.java PRE-CREATION
>   flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java PRE-CREATION
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 953a670
>
> Diff: https://reviews.apache.org/r/6377/diff/
>
>
> Testing
> -------
>
> Extensive unit tests and I also built and played with this using a stub flume agent. If you look at the JIRA I have a configuration file for an agent that will print out Avro events to the command line - that's helpful when testing this.
>
>
> Thanks,
>
> Patrick Wendell
>
>

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6377/
-----------------------------------------------------------

(Updated Nov. 6, 2012, 6:34 a.m.)
Review request for Flume.
Changes
-------

This is a small patch that should fix the unit test issues. I need someone to run on Mac to confirm that this works.
Description
-------

This patch adds a spooling directory based source. The  idea is that a user can have a spool directory where files are deposited for ingestion into flume. Once ingested, the files are clearly renamed and the implementation guarantees at-least-once delivery semantics similar to those achieved within flume itself, even across failures and restarts of the JVM running the code.

This helps fill the gap for people who want a way to get reliable delivery of events into flume, but don't want to directly write their application against the flume API. They can simply drop log files off in a spooldir and let flume ingest asynchronously (using some shell scripts or other automated process).

Unlike the prior iteration, this patch implements a first-class source. It also extends the avro client to support spooling in a similar manner.
This addresses bug FlUME-1425.
    https://issues.apache.org/jira/browse/FlUME-1425
Diffs (updated)
-----

  flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceConfiguration.java da804d7
  flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java abbbf1c
  flume-ng-core/src/main/java/org/apache/flume/client/avro/AvroCLIClient.java 4a5ecae
  flume-ng-core/src/main/java/org/apache/flume/client/avro/BufferedLineReader.java PRE-CREATION
  flume-ng-core/src/main/java/org/apache/flume/client/avro/LineReader.java PRE-CREATION
  flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java PRE-CREATION
  flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java PRE-CREATION
  flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java PRE-CREATION
  flume-ng-core/src/test/java/org/apache/flume/client/avro/TestBufferedLineReader.java PRE-CREATION
  flume-ng-core/src/test/java/org/apache/flume/client/avro/TestSpoolingFileLineReader.java PRE-CREATION
  flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java PRE-CREATION
  flume-ng-doc/sphinx/FlumeUserGuide.rst 953a670

Diff: https://reviews.apache.org/r/6377/diff/
Testing
-------

Extensive unit tests and I also built and played with this using a stub flume agent. If you look at the JIRA I have a configuration file for an agent that will print out Avro events to the command line - that's helpful when testing this.
Thanks,

Patrick Wendell

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6377/#review13130
-----------------------------------------------------------

Ship it!
Patch applied, OSX ML

------------------------------------------------------
 T E S T S
-------------------------------------------------------

-------------------------------------------------------
 T E S T S
-------------------------------------------------------
Running org.apache.flume.api.TestFailoverRpcClient
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 9.597 sec
Running org.apache.flume.api.TestLoadBalancingRpcClient
Tests run: 13, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 33.793 sec
Running org.apache.flume.api.TestNettyAvroRpcClient
Tests run: 12, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.845 sec
Running org.apache.flume.api.TestRpcClientFactory
Tests run: 6, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.279 sec
Running org.apache.flume.event.TestEventBuilder
Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.045 sec

Results :

Tests run: 35, Failures: 0, Errors: 0, Skipped: 0

- Alexander Alten-Lorenz
On Nov. 6, 2012, 6:34 a.m., Patrick Wendell wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6377/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2012, 6:34 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This patch adds a spooling directory based source. The  idea is that a user can have a spool directory where files are deposited for ingestion into flume. Once ingested, the files are clearly renamed and the implementation guarantees at-least-once delivery semantics similar to those achieved within flume itself, even across failures and restarts of the JVM running the code.
>
> This helps fill the gap for people who want a way to get reliable delivery of events into flume, but don't want to directly write their application against the flume API. They can simply drop log files off in a spooldir and let flume ingest asynchronously (using some shell scripts or other automated process).
>
> Unlike the prior iteration, this patch implements a first-class source. It also extends the avro client to support spooling in a similar manner.
>
>
> This addresses bug FlUME-1425.
>     https://issues.apache.org/jira/browse/FlUME-1425
>
>
> Diffs
> -----
>
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceConfiguration.java da804d7
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java abbbf1c
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/AvroCLIClient.java 4a5ecae
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/BufferedLineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/LineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java PRE-CREATION
>   flume-ng-core/src/test/java/org/apache/flume/client/avro/TestBufferedLineReader.java PRE-CREATION
>   flume-ng-core/src/test/java/org/apache/flume/client/avro/TestSpoolingFileLineReader.java PRE-CREATION
>   flume-ng-core/src/test/java/org/apache/flume/source/TestSpoolDirectorySource.java PRE-CREATION
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 953a670
>
> Diff: https://reviews.apache.org/r/6377/diff/
>
>
> Testing
> -------
>
> Extensive unit tests and I also built and played with this using a stub flume agent. If you look at the JIRA I have a configuration file for an agent that will print out Avro events to the command line - that's helpful when testing this.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/6377/#review13136
-----------------------------------------------------------

flume-ng-core/src/test/java/org/apache/flume/client/avro/TestSpoolingFileLineReader.java
<https://reviews.apache.org/r/6377/#comment28292>

    fails:
    testBehaviorWithEmptyFile(org.apache.flume.client.avro.TestSpoolingFileLineReader)  Time elapsed: 0.005 sec  <<< FAILURE!
    java.lang.AssertionError
            at org.junit.Assert.fail(Assert.java:92)
            at org.junit.Assert.assertTrue(Assert.java:43)
            at org.junit.Assert.assertTrue(Assert.java:54)
            at org.apache.flume.client.avro.TestSpoolingFileLineReader.testBehaviorWithEmptyFile(TestSpoolingFileLineReader.java:396)

flume-ng-core/src/test/java/org/apache/flume/client/avro/TestSpoolingFileLineReader.java
<https://reviews.apache.org/r/6377/#comment28293>

    fails:
    2012-11-06 10:04:43,339 (main) [ERROR - org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:196)] Found line longer than 15 characters, cannot make progress.
    2012-11-06 10:04:43,339 (main) [ERROR - org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:200)] Invalid line starts with: reallyreallyreallyreallyreally
    2012-11-06 10:04:43,342 (main) [ERROR - org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:196)] Found line longer than 15 characters, cannot make progress.
    2012-11-06 10:04:43,342 (main) [ERROR - org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:200)] Invalid line starts with: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    2012-11-06 10:04:43,344 (main) [ERROR - org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:196)] Found line longer than 15 characters, cannot make progress.
    2012-11-06 10:04:43,344 (main) [ERROR - org.apache.flume.client.avro.SpoolingFileLineReader.readLines(SpoolingFileLineReader.java:200)] Invalid line starts with: xxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
- Alexander Alten-Lorenz
On Nov. 6, 2012, 6:34 a.m., Patrick Wendell wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/6377/
> -----------------------------------------------------------
>
> (Updated Nov. 6, 2012, 6:34 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This patch adds a spooling directory based source. The  idea is that a user can have a spool directory where files are deposited for ingestion into flume. Once ingested, the files are clearly renamed and the implementation guarantees at-least-once delivery semantics similar to those achieved within flume itself, even across failures and restarts of the JVM running the code.
>
> This helps fill the gap for people who want a way to get reliable delivery of events into flume, but don't want to directly write their application against the flume API. They can simply drop log files off in a spooldir and let flume ingest asynchronously (using some shell scripts or other automated process).
>
> Unlike the prior iteration, this patch implements a first-class source. It also extends the avro client to support spooling in a similar manner.
>
>
> This addresses bug FlUME-1425.
>     https://issues.apache.org/jira/browse/FlUME-1425
>
>
> Diffs
> -----
>
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceConfiguration.java da804d7
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/source/SourceType.java abbbf1c
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/AvroCLIClient.java 4a5ecae
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/BufferedLineReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/LineReader.java PRE-CREATION