Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Re: Review Request 11583: HDFS Sink should check if file is closed and retry if it is not.


+
Ted Malaska 2013-06-28, 14:29
Copy link to this message
-
Re: Review Request 11583: HDFS Sink should check if file is closed and retry if it is not.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/11583/
-----------------------------------------------------------

(Updated June 28, 2013, 2:29 p.m.)
Review request for Flume.
Changes
-------

Added Hari's review changes
Bugs: 2007
    https://issues.apache.org/jira/browse/2007
Repository: flume-git
Description
-------

We can use the new API added in HDFS-4525. We will need to use reflection though, so we can run against a version of HDFS which does not have this API.
Diffs
-----

  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/AbstractHDFSWriter.java bc3b383
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java 2c2be6a
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java b8214be
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java 0383744
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFileSystemCloseRetryWrapper.java PRE-CREATION
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockFsDataOutputStreamCloseRetryWrapper.java PRE-CREATION
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestUseRawLocalFileSystem.java ffbdde0

Diff: https://reviews.apache.org/r/11583/diff/
Testing
-------
File Attachments
----------------

Patch-0
  https://reviews.apache.org/media/uploaded/files/2013/05/31/FLUME-2007-0.patch
Thanks,

Ted Malaska

+
Hari Shreedharan 2013-07-01, 21:47
+
Ted Malaska 2013-07-02, 13:00
+
Ted Malaska 2013-07-02, 13:18
+
Hari Shreedharan 2013-07-10, 00:24
+
Hari Shreedharan 2013-07-10, 00:29
+
Ted Malaska 2013-07-10, 02:26
+
Ted Malaska 2013-07-10, 02:26
+
Hari Shreedharan 2013-07-10, 19:25
+
Ted Malaska 2013-07-11, 01:28
+
Hari Shreedharan 2013-07-11, 02:52
+
Hari Shreedharan 2013-07-11, 02:19