Thanks for the hints!
The part that you quoted is in fact from the flume wiki and I forgot to
quote it myself. But the page is quite old so I wasn't sure about the state
of the feature. I thought maybe commons-configuration was used but wasn't
be able to confirm it clearly (or more exactly I lost myself in this
dependency, which might be used by one of my current projects somehow).
> Flume NG has a pluggable configuration system called the configuration
> provider. By default, Flume NG ships with a Java property file based
> configuration system that is both simple and easy to generate
> programmatically. Flume OG has a centralized configuration system with
> a master and ZooKeeper for coordination and we recognize this is very
> appealing to some users where as others see it as overhead they simply
> don't want. We opted to make this a pluggable extension point and ship
> a basic implementation that would let many users get started quickly
> and easily. There's almost certainly enough desire for a similar
> implementation to that of Flume OG, but it isn't yet implemented.
> Users may also implement arbitrary plugins to integrate with any type
> of configuration system (JSON files, a shared RDBMS, a central system
> with ZooKeeper, and so forth). We see this as something more
> interesting to system integrators.
I will look at the classes you pointed for my own education and I will see
what I can do.
Thanks a lot.
On Fri, Mar 29, 2013 at 4:19 PM, Brock Noland <[EMAIL PROTECTED]> wrote:
> "Flume NG has a pluggable configuration system called the configuration
> There is an object hierarchy, but I don't consider it pluggable as there
> is no place to "plugin" your own implementation. I consider these things
> "private" to Flume. Meaning that there is no guaranteed API stability
> between releases. With that caveat, you'd want to implement the following
> abstract class:
> and then'd you have to customize the Application class to "plugin" your
> own implementation:
> On Fri, Mar 29, 2013 at 8:53 AM, Bertrand Dechoux <[EMAIL PROTECTED]>wrote:
>> Flume NG has a pluggable configuration system called the configuration
>> provider. I cloned the source repository and I indeed found it but I am a
>> bit lost. Is there some kind of short guide explaining how to write a
>> custom configuration provider?
>> I would like to start with something simple : using a YAML configuration
>> file instead of a properties file.
>> It should be almost trivial to change the following configuration
>> *agent1.channels.ch1.type = memory
>> agent1.sources.avro-source1.channels = ch1
>> agent1.sources.avro-source1.type = avro
>> agent1.sources.avro-source1.bind = 0.0.0.0
>> agent1.sources.avro-source1.port = 41414
>> agent1.sinks.log-sink1.channel = ch1
>> agent1.sinks.log-sink1.type = logger
>> agent1.channels = ch1
>> agent1.sources = avro-source1
>> agent1.sinks = log-sink1*
>> type: memory
>> channels: ch1
>> type: avro
>> bind: 0.0.0.0
>> port: 41414
>> channel: ch1
>> type: logger
>> At least, I know how to convert the latter into the former but I am not
>> sure how this transformation can be included into flume-ng using a
>> configuration provider.
>> I have not been able to find much information about the customisation of