Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Short guide to custom configuration provider?


Copy link to this message
-
Short guide to custom configuration provider?
Hi,

Flume NG has a pluggable configuration system called the configuration
provider. I cloned the source repository and I indeed found it but I am a
bit lost. Is there some kind of short guide explaining how to write a
custom configuration provider?

I would like to start with something simple : using a YAML configuration
file instead of a properties file.
It should be almost trivial to change the following configuration

*agent1.channels.ch1.type = memory

agent1.sources.avro-source1.channels = ch1
agent1.sources.avro-source1.type = avro
agent1.sources.avro-source1.bind = 0.0.0.0
agent1.sources.avro-source1.port = 41414

agent1.sinks.log-sink1.channel = ch1
agent1.sinks.log-sink1.type = logger

agent1.channels = ch1
agent1.sources = avro-source1
agent1.sinks = log-sink1*

into

agent1:

    channels:
        ch1:
            type: memory

    sources:
        avroSource1:
            channels: ch1
            type: avro
            bind: 0.0.0.0
            port: 41414

    sinks:
        logSink1:
            channel: ch1
            type: logger
At least, I know how to convert the latter into the former but I am not
sure how this transformation can be included into flume-ng using a
configuration provider.
I have not been able to find much information about the customisation of
the configuration provider. Did I miss any documentation?
Bertrand
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB