Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Review Request: FLUME-1916: HDFS sink should poll for # of active replicas. If less than required, roll the file


+
Mike Percy 2013-02-28, 06:52
+
Hari Shreedharan 2013-02-28, 23:55
+
Mike Percy 2013-03-01, 01:55
+
Mike Percy 2013-03-01, 01:53
+
Hari Shreedharan 2013-03-01, 04:47
+
Mike Percy 2013-03-01, 05:45
+
Mike Percy 2013-03-01, 05:44
+
Hari Shreedharan 2013-03-01, 05:58
+
Mike Percy 2013-03-01, 06:40
+
Mike Percy 2013-03-01, 06:38
Copy link to this message
-
Re: Review Request: FLUME-1916: HDFS sink should poll for # of active replicas. If less than required, roll the file

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/9675/#review17236
-----------------------------------------------------------

Ship it!
Ship It!

- Hari Shreedharan
On March 1, 2013, 6:38 a.m., Mike Percy wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/9675/
> -----------------------------------------------------------
>
> (Updated March 1, 2013, 6:38 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> This is based on https://issues.apache.org/jira/browse/HBASE-2234 ... essentially if datanodes are dying en masse then we want to close our files as we are writing so that we get new pipeline assignments from the namenode.
>
>
> This addresses bug FLUME-1916.
>     https://issues.apache.org/jira/browse/FLUME-1916
>
>
> Diffs
> -----
>
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 5ac903e
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/AbstractHDFSWriter.java PRE-CREATION
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java cdc37f6
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java 1f3521e
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java 4ea4327
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java 3bd25f4
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSWriter.java abca21f
>   flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/MockHDFSWriter.java 5e8628b
>   flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestBucketWriter.java 99b6150
>   flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSinkOnMiniCluster.java 2e71069
>
> Diff: https://reviews.apache.org/r/9675/diff/
>
>
> Testing
> -------
>
> Added unit test.
>
>
> Thanks,
>
> Mike Percy
>
>