Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request 13640: HDFS Sink rollSize is calculated based off of uncompressed size of cumulative events.


Copy link to this message
-
Re: Review Request 13640: HDFS Sink rollSize is calculated based off of uncompressed size of cumulative events.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/13640/
-----------------------------------------------------------

(Updated Nov. 5, 2013, 4:53 a.m.)
Review request for Flume.
Changes
-------

First implementation of Mike's design
Bugs: 2128
    https://issues.apache.org/jira/browse/2128
Repository: flume-git
Description
-------

1) Added support for hdfs.rollCompressedSize
2) Added unit test
3) Updated doc
Diffs (updated)
-----

  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/AbstractHDFSWriter.java da0466d
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java 65f4d2c
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java 5518547
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java e20d1ee
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSEventSink.java f0a6e4b
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java 5fe9f1b
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSWriter.java 44a984a
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/HDFSBadSeqWriter.java 63ab5af
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockHDFSWriter.java ec49b97
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java f741e03

Diff: https://reviews.apache.org/r/13640/diff/
Testing
-------
Thanks,

Ted Malaska