-Re: Short guide to custom configuration provider?
Bertrand Dechoux 2013-03-29, 16:43
It seems quite easier than dealing with commons configuration.
I might take a shot at it during the week-end, if I find the time.
About the customisation of the Application class, that's something which is
already mostly done by
https://issues.apache.org/jira/browse/FLUME-1491 : Dynamic Configuration
from Zookeeper watcher
It might be a good idea to split this improvement into :
1) Selection of an alternative configuration provider
2) A zookeeper based configuration provider
If point 1 is done, anyone could use her/his custom configuration provider
by dropping the implementation as a third party jar and then selecting it
with the help of the command line.
On Fri, Mar 29, 2013 at 4:46 PM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:
> Thanks for the hints!
> The part that you quoted is in fact from the flume wiki and I forgot to
> quote it myself. But the page is quite old so I wasn't sure about the state
> of the feature. I thought maybe commons-configuration was used but wasn't
> be able to confirm it clearly (or more exactly I lost myself in this
> dependency, which might be used by one of my current projects somehow).
> Configuration Provider
>> Flume NG has a pluggable configuration system called the configuration
>> provider. By default, Flume NG ships with a Java property file based
>> configuration system that is both simple and easy to generate
>> programmatically. Flume OG has a centralized configuration system with
>> a master and ZooKeeper for coordination and we recognize this is very
>> appealing to some users where as others see it as overhead they simply
>> don't want. We opted to make this a pluggable extension point and ship
>> a basic implementation that would let many users get started quickly
>> and easily. There's almost certainly enough desire for a similar
>> implementation to that of Flume OG, but it isn't yet implemented.
>> Users may also implement arbitrary plugins to integrate with any type
>> of configuration system (JSON files, a shared RDBMS, a central system
>> with ZooKeeper, and so forth). We see this as something more
>> interesting to system integrators.
> I will look at the classes you pointed for my own education and I will see
> what I can do.
> Thanks a lot.
> On Fri, Mar 29, 2013 at 4:19 PM, Brock Noland <[EMAIL PROTECTED]> wrote:
>> "Flume NG has a pluggable configuration system called the configuration
>> There is an object hierarchy, but I don't consider it pluggable as there
>> is no place to "plugin" your own implementation. I consider these things
>> "private" to Flume. Meaning that there is no guaranteed API stability
>> between releases. With that caveat, you'd want to implement the following
>> abstract class:
>> and then'd you have to customize the Application class to "plugin" your
>> own implementation:
>> On Fri, Mar 29, 2013 at 8:53 AM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:
>>> Flume NG has a pluggable configuration system called the configuration
>>> provider. I cloned the source repository and I indeed found it but I am a
>>> bit lost. Is there some kind of short guide explaining how to write a
>>> custom configuration provider?
>>> I would like to start with something simple : using a YAML configuration
>>> file instead of a properties file.
>>> It should be almost trivial to change the following configuration
>>> *agent1.channels.ch1.type = memory
>>> agent1.sources.avro-source1.channels = ch1