Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Short guide to custom configuration provider?


Copy link to this message
-
Short guide to custom configuration provider?
Bertrand Dechoux 2013-03-29, 13:53
Hi,

Flume NG has a pluggable configuration system called the configuration
provider. I cloned the source repository and I indeed found it but I am a
bit lost. Is there some kind of short guide explaining how to write a
custom configuration provider?

I would like to start with something simple : using a YAML configuration
file instead of a properties file.
It should be almost trivial to change the following configuration

*agent1.channels.ch1.type = memory

agent1.sources.avro-source1.channels = ch1
agent1.sources.avro-source1.type = avro
agent1.sources.avro-source1.bind = 0.0.0.0
agent1.sources.avro-source1.port = 41414

agent1.sinks.log-sink1.channel = ch1
agent1.sinks.log-sink1.type = logger

agent1.channels = ch1
agent1.sources = avro-source1
agent1.sinks = log-sink1*

into

agent1:

    channels:
        ch1:
            type: memory

    sources:
        avroSource1:
            channels: ch1
            type: avro
            bind: 0.0.0.0
            port: 41414

    sinks:
        logSink1:
            channel: ch1
            type: logger
At least, I know how to convert the latter into the former but I am not
sure how this transformation can be included into flume-ng using a
configuration provider.
I have not been able to find much information about the customisation of
the configuration provider. Did I miss any documentation?
Bertrand
+
Brock Noland 2013-03-29, 15:19
+
Bertrand Dechoux 2013-03-29, 15:46
+
Bertrand Dechoux 2013-03-29, 16:43
+
Hari Shreedharan 2013-03-29, 19:06
+
Bertrand Dechoux 2013-04-02, 19:23
+
Hari Shreedharan 2013-04-02, 20:34