Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # dev - Review Request: FLUME-1632: Persist progress on each file in file spooling client/source


+
Mike Percy 2012-12-14, 09:56
+
Brock Noland 2012-12-14, 15:22
+
Mike Percy 2012-12-16, 13:31
+
Mike Percy 2012-12-16, 13:31
+
Mike Percy 2012-12-16, 13:33
+
Mike Percy 2012-12-17, 08:58
+
Brock Noland 2012-12-18, 17:44
+
Mike Percy 2012-12-18, 21:25
+
Brock Noland 2012-12-18, 21:29
+
Mike Percy 2012-12-18, 21:50
+
Mike Percy 2012-12-18, 23:22
+
Brock Noland 2012-12-18, 23:42
+
Brock Noland 2012-12-19, 00:00
+
Brock Noland 2012-12-19, 01:54
+
Mike Percy 2012-12-19, 12:41
+
Mike Percy 2012-12-19, 12:38
+
Brock Noland 2012-12-19, 15:35
+
Brock Noland 2012-12-19, 17:40
Copy link to this message
-
Re: Review Request: FLUME-1632: Persist progress on each file in file spooling client/source
Mike Percy 2012-12-20, 03:10


> On Dec. 19, 2012, 3:35 p.m., Brock Noland wrote:
> > Thanks for updating the patch! I am getting these two test failures:
> >
> >
> > Tests in error:
> >   testGoodTrackerFile(org.apache.flume.serialization.TestDurablePositionTracker)
> >   testPartialTrackerFile(org.apache.flume.serialization.TestDurablePositionTracker)
>
> Brock Noland wrote:
>     This is because the included .avro files did not apply with the patch command. git apply fixes this.

Yes, you need the latest version of GNU patch if you need Git binary patch support in the patch utility. Thanks Brock!!
- Mike
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8596/#review14720
-----------------------------------------------------------
On Dec. 19, 2012, 12:37 p.m., Mike Percy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8596/
> -----------------------------------------------------------
>
> (Updated Dec. 19, 2012, 12:37 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Defines EventDeserializer interface and uses it from the spooling source. Progress is persisted as bytes are read from the underlying file.
>
>
> This addresses bug FLUME-1632.
>     https://issues.apache.org/jira/browse/FLUME-1632
>
>
> Diffs
> -----
>
>   flume-ng-core/pom.xml 0224519
>   flume-ng-core/src/main/avro/TransferStateFileMeta.avsc PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/AvroCLIClient.java 37e9ffa
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/BufferedLineReader.java 718e1b2
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/EventReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/LineReader.java 904f22c
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableEventReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/ReliableSpoolingFileEventReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/SimpleTextLineEventReader.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/client/avro/SpoolingFileLineReader.java 8362299
>   flume-ng-core/src/main/java/org/apache/flume/serialization/DurablePositionTracker.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventDeserializer.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventDeserializerFactory.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventDeserializerType.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerDe.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializer.java a418935
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializerFactory.java 75853a9
>   flume-ng-core/src/main/java/org/apache/flume/serialization/EventSerializerType.java afe8ed8
>   flume-ng-core/src/main/java/org/apache/flume/serialization/LineDeserializer.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/PositionTracker.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/Resettable.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/ResettableFileInputStream.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/serialization/ResettableInputStream.java PRE-CREATION
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySource.java 61824d8
>   flume-ng-core/src/main/java/org/apache/flume/source/SpoolDirectorySourceConfigurationConstants.java 806a661
>   flume-ng-core/src/main/java/org/apache/flume/tools/PlatformDetect.java PRE-CREATION
>   flume-ng-core/src/test/java/org/apache/flume/client/avro/TestBufferedLineReader.java 169abe5
+
Brock Noland 2012-12-19, 18:12