-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8964/
-----------------------------------------------------------

Review request for Flume.
Description
-------

Add's flag to HDFSEventSink Serializer's to use raw local file system.
This addresses bug FLUME-1844.
    https://issues.apache.org/jira/browse/FLUME-1844
Diffs
-----

  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java f14f7cb
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java 18fe6d4
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java bd40a88
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java 1e6d68f
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java 1035ac3
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestUseRawLocalFileSystem.java PRE-CREATION

Diff: https://reviews.apache.org/r/8964/diff/
Testing
-------

Unit tests pass.
Thanks,

Brock Noland

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8964/
-----------------------------------------------------------

(Updated Jan. 15, 2013, 10:33 p.m.)
Review request for Flume, Juhani Connolly and Hari Shreedharan.
Changes
-------

Added specific reviewers.
Description
-------

Add's flag to HDFSEventSink Serializer's to use raw local file system.
This addresses bug FLUME-1844.
    https://issues.apache.org/jira/browse/FLUME-1844
Diffs
-----

  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java f14f7cb
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java 18fe6d4
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java bd40a88
  flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java 1e6d68f
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java 1035ac3
  flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestUseRawLocalFileSystem.java PRE-CREATION

Diff: https://reviews.apache.org/r/8964/diff/
Testing
-------

Unit tests pass.
Thanks,

Brock Noland

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8964/#review15361
-----------------------------------------------------------
Brock,

Looks good in general. One question I have is if we could just do without a flume flag. Rather check the Configuration object to see if it is set to write to local file system. Does that make sense?

- Hari Shreedharan
On Jan. 15, 2013, 10:33 p.m., Brock Noland wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8964/
> -----------------------------------------------------------
>
> (Updated Jan. 15, 2013, 10:33 p.m.)
>
>
> Review request for Flume, Juhani Connolly and Hari Shreedharan.
>
>
> Description
> -------
>
> Add's flag to HDFSEventSink Serializer's to use raw local file system.
>
>
> This addresses bug FLUME-1844.
>     https://issues.apache.org/jira/browse/FLUME-1844
>
>
> Diffs
> -----
>
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java f14f7cb
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java 18fe6d4
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java bd40a88
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java 1e6d68f
>   flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java 1035ac3
>   flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestUseRawLocalFileSystem.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/8964/diff/
>
>
> Testing
> -------
>
> Unit tests pass.
>
>
> Thanks,
>
> Brock Noland
>
>

> On Jan. 16, 2013, 12:21 a.m., Hari Shreedharan wrote:
> > Brock,
> >
> > Looks good in general. One question I have is if we could just do without a flume flag. Rather check the Configuration object to see if it is set to write to local file system. Does that make sense?
>
> Brock Noland wrote:
>     I thought about that as well. The reason I did not do that is because file:/// implements ChecksumFileSystem so data you read/write is checksummed. We could probably change it and no one would notice but it's possible someone is using this feature.... Thoughts?

Makes sense. +1
- Hari
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8964/#review15361
-----------------------------------------------------------
On Jan. 15, 2013, 10:33 p.m., Brock Noland wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8964/
> -----------------------------------------------------------
>
> (Updated Jan. 15, 2013, 10:33 p.m.)
>
>
> Review request for Flume, Juhani Connolly and Hari Shreedharan.
>
>
> Description
> -------
>
> Add's flag to HDFSEventSink Serializer's to use raw local file system.
>
>
> This addresses bug FLUME-1844.
>     https://issues.apache.org/jira/browse/FLUME-1844
>
>
> Diffs
> -----
>
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java f14f7cb
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java 18fe6d4
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java bd40a88
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java 1e6d68f
>   flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java 1035ac3
>   flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestUseRawLocalFileSystem.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/8964/diff/
>
>
> Testing
> -------
>
> Unit tests pass.
>
>
> Thanks,
>
> Brock Noland
>
>

> On Jan. 16, 2013, 12:21 a.m., Hari Shreedharan wrote:
> > Brock,
> >
> > Looks good in general. One question I have is if we could just do without a flume flag. Rather check the Configuration object to see if it is set to write to local file system. Does that make sense?

I thought about that as well. The reason I did not do that is because file:/// implements ChecksumFileSystem so data you read/write is checksummed. We could probably change it and no one would notice but it's possible someone is using this feature.... Thoughts?
- Brock
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/8964/#review15361
-----------------------------------------------------------
On Jan. 15, 2013, 10:33 p.m., Brock Noland wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/8964/
> -----------------------------------------------------------
>
> (Updated Jan. 15, 2013, 10:33 p.m.)
>
>
> Review request for Flume, Juhani Connolly and Hari Shreedharan.
>
>
> Description
> -------
>
> Add's flag to HDFSEventSink Serializer's to use raw local file system.
>
>
> This addresses bug FLUME-1844.
>     https://issues.apache.org/jira/browse/FLUME-1844
>
>
> Diffs
> -----
>
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/BucketWriter.java f14f7cb
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSCompressedDataStream.java 18fe6d4
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSDataStream.java bd40a88
>   flume-ng-sinks/flume-hdfs-sink/src/main/java/org/apache/flume/sink/hdfs/HDFSSequenceFile.java 1e6d68f
>   flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestHDFSEventSink.java 1035ac3
>   flume-ng-sinks/flume-hdfs-sink/src/test/java/org/apache/flume/sink/hdfs/TestUseRawLocalFileSystem.java PRE-CREATION
>
> Diff: https://reviews.apache.org/r/8964/diff/
>
>
> Testing
> -------
>
> Unit tests pass.
>
>
> Thanks,
>
> Brock Noland
>
>