Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request: FLUME-1630: Flume configuration code could be improved

Copy link to this message
Re: Review Request: FLUME-1630: Flume configuration code could be improved

> On Nov. 20, 2012, 6:55 p.m., Hari Shreedharan wrote:
> > The patch does not apply cleanly. Could you please rebase?
> Brock Noland wrote:
>     It was committed to the FLUME-1502 branch which, as I understand it, was so I didn't have to rebase it all the time?

I was trying to apply to trunk, I forgot about the branch. So the branch has only this patch right, not the 1502 patch too?
- Hari
This is an automatically generated e-mail. To reply, visit:
On Oct. 11, 2012, 6:58 p.m., Brock Noland wrote:
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/7518/
> -----------------------------------------------------------
> (Updated Oct. 11, 2012, 6:58 p.m.)
> Review request for Flume.
> Description
> -------
> Patch is not for commit, yet. It changes the configuration system into something extendable and maintainable. Additionally it changes the terminology from node to agent.  Once the patch is ready for review we should change the node package to agent to conform to the agent terminology.
> Big ticket items:
> 1) Abstract property file provider is changed to Abstract property provider. Two concrete implementations are provided, PropertyFileConfigurationProvider and PollingPropertyFileConfigurationProvider. There is an additional concrete implementation MemoryConfigurationProvider is in TestAbstractConfigurationProvider.
> 2) Caching instances is removed from all factories. Instance caching is implemented in AbstractConfigurationProvider for channels *if* they have the Reusable annotation. MemoryChannel has this annotation.
> 3) A layer of supervisors is removed. The application class now starts and stops the components when handleConfigurationEvent is called. This is called on startup if PropertyFileConfigurationProvider is used or whenever the configuration file changes if PollingPropertyFileConfigurationProvider is used. PollingPropertyFileConfigurationProvider uses EventBus (guava) to trigger the re-configuration.
> This addresses bug FLUME-1630.
>     https://issues.apache.org/jira/browse/FLUME-1630
> Diffs
> -----
>   flume-ng-channels/flume-file-channel/src/main/java/org/apache/flume/channel/file/FileChannel.java 6680a2c
>   flume-ng-channels/flume-jdbc-channel/src/main/java/org/apache/flume/channel/jdbc/JdbcChannel.java bca0c50
>   flume-ng-channels/flume-recoverable-memory-channel/src/main/java/org/apache/flume/channel/recoverable/memory/RecoverableMemoryChannel.java 49e7cfd
>   flume-ng-configuration/src/main/java/org/apache/flume/conf/FlumeConfiguration.java 9b209e8
>   flume-ng-core/src/main/java/org/apache/flume/ChannelFactory.java e269909
>   flume-ng-core/src/main/java/org/apache/flume/Constants.java 4c6992d
>   flume-ng-core/src/main/java/org/apache/flume/SinkFactory.java d8fd0da
>   flume-ng-core/src/main/java/org/apache/flume/SourceFactory.java 91cc866
>   flume-ng-core/src/main/java/org/apache/flume/channel/DefaultChannelFactory.java ee32696
>   flume-ng-core/src/main/java/org/apache/flume/channel/MemoryChannel.java fc3a1e2
>   flume-ng-core/src/main/java/org/apache/flume/sink/DefaultSinkFactory.java e71d44e
>   flume-ng-core/src/main/java/org/apache/flume/source/DefaultSourceFactory.java 18533ae
>   flume-ng-core/src/test/java/org/apache/flume/sink/TestDefaultSinkFactory.java ab3f447
>   flume-ng-core/src/test/java/org/apache/flume/source/TestDefaultSourceFactory.java ff5b4d6
>   flume-ng-node/pom.xml 5464bd3
>   flume-ng-node/src/main/java/org/apache/flume/conf/file/AbstractFileConfigurationProvider.java a2c882b
>   flume-ng-node/src/main/java/org/apache/flume/conf/file/SimpleNodeConfiguration.java 99b8bcc
>   flume-ng-node/src/main/java/org/apache/flume/conf/properties/PropertiesFileConfigurationProvider.java 8dbbe57