Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # dev - Review Request: FLUME-1586. File Channel should support verifying integrity of individual events.


+
Hari Shreedharan 2013-05-06, 07:49
+
Hari Shreedharan 2013-05-06, 20:12
+
Hari Shreedharan 2013-05-06, 21:28
+
Hari Shreedharan 2013-05-23, 01:52
+
Hari Shreedharan 2013-05-23, 19:59
+
Hari Shreedharan 2013-05-23, 20:54
Copy link to this message
-
Re: Review Request: FLUME-1586. File Channel should support verifying integrity of individual events.
Brock Noland 2013-05-23, 22:01

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10944/#review20956
-----------------------------------------------------------
Hari, this is great!  I have a few questions/comments below. I just did an initial pass. Also since we are making so many classes public should we add visibility annotations to those classes?
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java
<https://reviews.apache.org/r/10944/#comment43254>

    Curious as to what byte this ends up representing?

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java
<https://reviews.apache.org/r/10944/#comment43255>

    this. is not consistent with the above

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java
<https://reviews.apache.org/r/10944/#comment43256>

    Let's log the offset and file like the message above.

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Serialization.java
<https://reviews.apache.org/r/10944/#comment43250>

    Curious why we have "" + ".old"?

flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannel.java
<https://reviews.apache.org/r/10944/#comment43253>

    This should be typed IllegalStateException no?

flume-tools/src/main/java/org/apache/flume/tools/FileChannelIntegrityTool.java
<https://reviews.apache.org/r/10944/#comment43262>

    Curious as to what case would make eventPosition == -1?
- Brock Noland
On May 23, 2013, 8:54 p.m., Hari Shreedharan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10944/
> -----------------------------------------------------------
>
> (Updated May 23, 2013, 8:54 p.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Patch to add a checksum to events and replace them with a noop event using a tool, if corrupt.
>
>
> This addresses bug FLUME-1586.
>     https://issues.apache.org/jira/browse/FLUME-1586
>
>
> Diffs
> -----
>
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/CorruptEventException.java PRE-CREATION
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java cc0d38a
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java 1918baa
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java d3db896
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFileV3.java d9a2a9b
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/NoopRecordException.java PRE-CREATION
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Put.java 4235a79
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/ReplayHandler.java fc47b23
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Serialization.java d6897e1
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/TransactionEventRecord.java 073042f
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/proto/ProtosFactory.java 4860ac2
>   flume-ng-channels/flume-file-channel/src/main/proto/filechannel.proto 1e668d2
>   flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannel.java 0f7d14d
>   flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestLog.java 54978f8
>   flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestLogFile.java bef22ef
>   flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestTransactionEventRecordV3.java f403422
+
Hari Shreedharan 2013-05-23, 23:52
+
Hari Shreedharan 2013-05-23, 22:48
+
Hari Shreedharan 2013-05-23, 23:14
+
Hari Shreedharan 2013-05-24, 00:00
+
Hari Shreedharan 2013-05-24, 04:59
+
Brock Noland 2013-06-12, 05:18
+
Hari Shreedharan 2013-06-12, 18:51