Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # dev - Review Request 16416: FLUME-2265. Closed bucket writers should be removed from sfwriters map


Copy link to this message
-
Review Request 16416: FLUME-2265. Closed bucket writers should be removed from sfwriters map
Hari Shreedharan 2013-12-20, 08:17

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/16416/
-----------------------------------------------------------

Review request for Flume.
Bugs: FLUME-2265
    https://issues.apache.org/jira/browse/FLUME-2265
Repository: flume-git
Description
-------

On idle or roll time based close, the bucket writer is closed and the bucket writer is removed from sfWriters map.
The patch introduces a new exception, which is thrown when an append tries to write to a bucket writer that is closed based on time (roll or idle).
The sink then catches this exception, creates a new bucket writer and writes to that one.
Diffs
-----

  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketClosedException.java PRE-CREATION
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java 200d457
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java f0a6e4b
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadSeqWriter.java 63ab5af
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadWriterFactory.java f5d0808
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSTestSeqWriter.java PRE-CREATION
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSTestWriterFactory.java PRE-CREATION
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java f741e03
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java 5b7cec9
  pom.xml 267925f

Diff: https://reviews.apache.org/r/16416/diff/
Testing
-------
Thanks,

Hari Shreedharan

+
Hari Shreedharan 2013-12-20, 08:18
+
Hari Shreedharan 2013-12-20, 17:58
+
Jarek Cecho 2013-12-23, 12:42
+
Jarek Cecho 2013-12-20, 15:59