Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Review Request: FLUME-1516. Write dual checkpoints.


+
Hari Shreedharan 2013-01-09, 07:06
+
Hari Shreedharan 2013-01-09, 17:09
+
Hari Shreedharan 2013-01-21, 21:36
+
Hari Shreedharan 2013-01-22, 00:43
+
Hari Shreedharan 2013-03-04, 20:44
+
Hari Shreedharan 2013-03-14, 07:31
+
Brock Noland 2013-03-14, 17:18
Copy link to this message
-
Re: Review Request: FLUME-1516. Write dual checkpoints.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8899/#review17888
-----------------------------------------------------------
Thanks for the review, Brock! I will submit an updated patch soon. Thanks!
flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
<https://reviews.apache.org/r/8899/#comment37865>

    Yes, Will do.

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
<https://reviews.apache.org/r/8899/#comment37866>

    Will update in next patch with a !null if condition

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
<https://reviews.apache.org/r/8899/#comment37867>

    The reason there is a backupComplete file is to ensure that we know that all files were copied. The idea is that if we were in the middle of copying (only some files were copied, not all), then this indicates the copying was incomplete. This would be for the case where the checkpoint is somehow corrupt and we try to retrieve the backup - at that point, we need to be sure the backup actually contains all files required (this makes it extensible, in case we add more files to the checkpoint)

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
<https://reviews.apache.org/r/8899/#comment37869>

    will fix

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
<https://reviews.apache.org/r/8899/#comment37871>

    will add check

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
<https://reviews.apache.org/r/8899/#comment37872>

    The previous backup is still happening, so we skip this checkpoint, rather than trying to abort the backup, cleanup the previous one and start the checkpoint. (If the checkpoint interval is too low, then there may never be a backup if we do that)

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
<https://reviews.apache.org/r/8899/#comment37873>

    Agreed

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java
<https://reviews.apache.org/r/8899/#comment37874>

    will do

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
<https://reviews.apache.org/r/8899/#comment37875>

    This was done to get the IDE to shut up ;)

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
<https://reviews.apache.org/r/8899/#comment37876>

    Yeah, this was just me being lazy :P

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
<https://reviews.apache.org/r/8899/#comment37877>

    yeah, I think we should

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java
<https://reviews.apache.org/r/8899/#comment37878>

    might make sense to do so.

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/LogFile.java
<https://reviews.apache.org/r/8899/#comment37879>

    yeah, this one confuses a lot of people!

flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Serialization.java
<https://reviews.apache.org/r/8899/#comment37880>

    will do
- Hari Shreedharan
On March 14, 2013, 7:31 a.m., Hari Shreedharan wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8899/
> -----------------------------------------------------------
>
> (Updated March 14, 2013, 7:31 a.m.)
>
>
> Review request for Flume.
+
Brock Noland 2013-03-14, 18:37
+
Hari Shreedharan 2013-03-14, 23:44
+
Brock Noland 2013-03-21, 03:01
+
Hari Shreedharan 2013-03-21, 21:30
+
Hari Shreedharan 2013-03-25, 21:03
+
Brock Noland 2013-03-25, 21:51
+
Hari Shreedharan 2013-03-25, 22:08
+
Brock Noland 2013-03-25, 22:12
+
Hari Shreedharan 2013-03-26, 22:52
+
Brock Noland 2013-04-04, 18:20
+
Hari Shreedharan 2013-04-04, 21:34
+
Hari Shreedharan 2013-04-04, 21:32
+
Hari Shreedharan 2013-04-05, 00:16
+
Brock Noland 2013-04-05, 14:35
+
Hari Shreedharan 2013-04-05, 18:01
+
Brock Noland 2013-04-05, 18:26