Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # dev - Review Request: FLUME-1632: Persist progress on each file in file spooling client/source


Copy link to this message
-
Re: Review Request: FLUME-1632: Persist progress on each file in file spooling client/source
Mike Percy 2012-12-16, 13:31


> On Dec. 14, 2012, 3:22 p.m., Brock Noland wrote:
> > flume-ng-clients/flume-ng-forwarder/src/main/java/org/apache/flume/clients/forwarder/FlumeTool.java, line 1
> > <https://reviews.apache.org/r/8596/diff/1/?file=238552#file238552line1>
> >
> >     Should this tool stuff be in core? Seems very useful for more than the forwarder...

I'm considering splitting this out into a separate patch soon, actually. It's not really baked.
> On Dec. 14, 2012, 3:22 p.m., Brock Noland wrote:
> > flume-ng-clients/flume-ng-forwarder/src/test/java/org/apache/flume/clients/forwarder/Blah2.java, line 1
> > <https://reviews.apache.org/r/8596/diff/1/?file=238555#file238555line1>
> >
> >     Probably a mistake?

Placeholder for the tests.
- Mike
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8596/#review14504
-----------------------------------------------------------
On Dec. 14, 2012, 9:56 a.m., Mike Percy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8596/
> -----------------------------------------------------------
>
> (Updated Dec. 14, 2012, 9:56 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Defines EventDeserializer interface and uses it from the spooling source. Progress is persisted as bytes are read from the underlying file.
>
>
> This addresses bug FLUME-1632.
>     https://issues.apache.org/jira/browse/FLUME-1632
>
>
> Diffs
> -----
>
>   flume-ng-clients/flume-ng-forwarder/pom.xml PRE-CREATION
>   flume-ng-clients/flume-ng-forwarder/src/main/java/org/apache/flume/clients/forwarder/FileClient.java PRE-CREATION
>   flume-ng-clients/flume-ng-forwarder/src/main/java/org/apache/flume/clients/forwarder/FlumeTool.java PRE-CREATION
>   flume-ng-clients/flume-ng-forwarder/src/main/java/org/apache/flume/clients/forwarder/ToolRegistry.java PRE-CREATION
>   flume-ng-clients/flume-ng-forwarder/src/main/java/org/apache/flume/clients/forwarder/ToolRunner.java PRE-CREATION
>   flume-ng-clients/flume-ng-forwarder/src/test/java/org/apache/flume/clients/forwarder/Blah2.java PRE-CREATION
>   flume-ng-clients/flume-ng-forwarder/src/test/resources/flume-log4jtest.properties PRE-CREATION
>   flume-ng-clients/pom.xml 7ea2d1d
>   flume-ng-core/pom.xml 2554a0e
>   flume-ng-core/src/main/avro/TransferStateFileMeta.avsc PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/AvroCLIClient.java 37e9ffa
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileEventParser.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java 8362299
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventDeserializer.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventDeserializerFactory.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventDeserializerType.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerDe.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializer.java a418935
>   flume-ng-core/src/main/java/org/apache/flume/serialization/LineDeserializer.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/PositionTracker.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/PositionTrackerFile.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/Resettable.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/ResettableFileInputStream.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/ResettableInputStream.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/Seekable.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 61824d8