Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request: FLUME-1285 FileChannel has a dependency on Hadoop IO classes


Copy link to this message
-
Review Request: FLUME-1285 FileChannel has a dependency on Hadoop IO classes

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10829/
-----------------------------------------------------------

Review request for Flume.
Description
-------

Modified and Applied Fix from [~chris.nagy] to 1.4 branch

Tested on Ubuntu 12.04 with 1.6.0_33 64-bit

modified:   flume-ng-channels/flume-file-channel/pom.xml
modified:   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEvent.java
modified:   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/TransactionEventRecord.java
new file:   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Writable.java
new file:   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/WritableUtils.java
modified:   flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestUtils.java
This addresses bug FLUME-1285.
    https://issues.apache.org/jira/browse/FLUME-1285
Diffs
-----

  flume-ng-channels/flume-file-channel/pom.xml 2408447
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEvent.java c447335
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/TransactionEventRecord.java 073042f
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Writable.java PRE-CREATION
  flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/WritableUtils.java PRE-CREATION
  flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestUtils.java 563dbcc

Diff: https://reviews.apache.org/r/10829/diff/
Testing
-------

Tested on Ubuntu 12.04 with Java 1.6.0_33 64-bit

- Unit Tests for Flume NG file-based channel were all successful with these changes.
- Created and deployed local build successfully with the following settings

agent.sources = seqGenSrc
agent.channels = fileChannel
agent.sinks = loggerSink
agent.sources.seqGenSrc.type = seq
agent.sources.seqGenSrc.channels = fileChannel

agent.channels.fileChannel.type = file
agent.channels.c1.checkpointDir = /flume-staging/file-channel/checkpoint
agent.channels.c1.dataDirs = /flume-staging/file-channel/data
agent.sinks.loggerSink.type = logger
agent.sinks.loggerSink.channel = fileChannel

Agent was started as follows

$ bin/flume-ng agent --conf-file conf/flume.properties --conf conf --name agent

Agent started up and loaded File Channel successfully.

Agent was also able to shut down successfully.
Thanks,

Israel Ekpo