Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # dev - Review Request: FLUME-1285 FileChannel has a dependency on Hadoop IO classes


+
Israel Ekpo 2013-04-29, 04:45
+
Roshan Naik 2013-05-17, 04:37
+
Israel Ekpo 2013-05-22, 06:20
+
Brock Noland 2013-05-23, 14:56
+
Israel Ekpo 2013-05-23, 19:35
+
Brock Noland 2013-05-23, 19:35
Copy link to this message
-
Re: Review Request: FLUME-1285 FileChannel has a dependency on Hadoop IO classes
Israel Ekpo 2013-05-23, 19:37


> On May 23, 2013, 2:56 p.m., Brock Noland wrote:
> > Hey this looks pretty good! Quick question, when you did the manual testing did you generate some data with the pre-patched version and then process the data with the post-patched version?
>
> Israel Ekpo wrote:
>     Hi Brock,
>    
>     I started from a clean slate.
>    
>     I built the code from scratch and then I started the agent with brand new events going from the source to the channel and then to the sink.
>    
>     Would you like me to build up events with the pre-patched version, stop the agent and then restart with the patched version?
>    
>
>
> Brock Noland wrote:
>     Hi,
>    
>     > Would you like me to build up events with the pre-patched version, stop the agent and then restart with the patched version?
>    
>     Yeah I don't expect any problem but it's probably something we should do...
>    
>

Ok,

I will work on that over the weekend.

Will update the review with my findings.
- Israel
-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10829/#review20939
-----------------------------------------------------------
On April 29, 2013, 4:45 a.m., Israel Ekpo wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10829/
> -----------------------------------------------------------
>
> (Updated April 29, 2013, 4:45 a.m.)
>
>
> Review request for Flume.
>
>
> Description
> -------
>
> Modified and Applied Fix from [~chris.nagy] to 1.4 branch
>
> Tested on Ubuntu 12.04 with 1.6.0_33 64-bit
>
> modified:   flume-ng-channels/flume-file-channel/pom.xml
> modified:   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEvent.java
> modified:   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/TransactionEventRecord.java
> new file:   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Writable.java
> new file:   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/WritableUtils.java
> modified:   flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestUtils.java
>
>
> This addresses bug FLUME-1285.
>     https://issues.apache.org/jira/browse/FLUME-1285
>
>
> Diffs
> -----
>
>   flume-ng-channels/flume-file-channel/pom.xml 2408447
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FlumeEvent.java c447335
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/TransactionEventRecord.java 073042f
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/Writable.java PRE-CREATION
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/WritableUtils.java PRE-CREATION
>   flume-ng-channels/flume-file-channel/src/test/java/org/apache/flume/channel/file/TestUtils.java 563dbcc
>
> Diff: https://reviews.apache.org/r/10829/diff/
>
>
> Testing
> -------
>
> Tested on Ubuntu 12.04 with Java 1.6.0_33 64-bit
>
> - Unit Tests for Flume NG file-based channel were all successful with these changes.
> - Created and deployed local build successfully with the following settings
>
> agent.sources = seqGenSrc
> agent.channels = fileChannel
> agent.sinks = loggerSink
>
>
> agent.sources.seqGenSrc.type = seq
> agent.sources.seqGenSrc.channels = fileChannel
>
> agent.channels.fileChannel.type = file
> agent.channels.c1.checkpointDir = /flume-staging/file-channel/checkpoint
> agent.channels.c1.dataDirs = /flume-staging/file-channel/data
>
>
> agent.sinks.loggerSink.type = logger
> agent.sinks.loggerSink.channel = fileChannel
>
> Agent was started as follows
>
> $ bin/flume-ng agent --conf-file conf/flume.properties --conf conf --name agent
>
> Agent started up and loaded File Channel successfully.