Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request: FLUME-1762. File Channel should recover automatically if the checkpoint is incomplete or bad by deleting the contents of the checkpoint directory


Copy link to this message
-
Re: Review Request: FLUME-1762. File Channel should recover automatically if the checkpoint is incomplete or bad by deleting the contents of the checkpoint directory

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8396/
-----------------------------------------------------------

(Updated Dec. 10, 2012, 5:40 p.m.)
Review request for Flume.
Changes
-------

Incorporating Brock's feedback.
Description
-------

Added code to throw a BadCheckpointException, if we can recover from the situation by deleting all files in the checkpoint directory. In the log class, during startup if BadCheckpointException is caught, all files are deleted and replay is retried.
This addresses bug FLUME-1762.
    https://issues.apache.org/jira/browse/FLUME-1762
Diffs (updated)
-----

  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/BadCheckpointException.java PRE-CREATION
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFactory.java 6c07152
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFile.java 5eaf8c2
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFileV2.java 8bbc081
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/EventQueueBackingStoreFileV3.java c24f89f
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEventQueue.java 36553c5
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Log.java 6d1cf51
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Serialization.java ef8cf72
  flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestEventQueueBackingStoreFactory.java b1a55be
  flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFileChannelRestart.java 3f90805
  flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestFlumeEventQueue.java 0173390

Diff: https://reviews.apache.org/r/8396/diff/
Testing
-------

Added unit tests. Modified some existing unit tests to test for this change.
Thanks,

Hari Shreedharan