Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Proposal for tools to generate duplicated configuration

Copy link to this message
Proposal for tools to generate duplicated configuration
hi all,
    In the progress of maintaining Flume NG, it's hard to manage
configuration in too many flows.
    Meanwhile, most of such configurations are similar. Maybe tiny
difference in component name;

    Thus, I would like to generate configuration with Java code in simple
    I created and tested tool to generate configuration. That runs

    Could anyone give your tips or advice? If tool like this is useful for
you, I will submit it to community.
    Wish your attention, thanks!
     Java Code Example:
                Agent agent = new Agent().setName("agent");
 Source source = new Source("scribe")
.setProperty("port", "1499")
.setType(SourceType.OTHER, "org.apache.flume.source.scribe.ScribeSource");
 for (int i = 0; i < 5; i++) {
Channel channel = new Channel("mc" + i)
      .setProperty("capacity", "1000000")
      .setType(ChannelType.MEMORY, null);
 Sink sink = new Sink("hfds" + i)
      .setType(SinkType.HDFS, null)
      .setProperty("hfds.txtEventMax", "1000")
       .setProperty("hdfs.path", "/flume_test/data" + i)

                ConfGenerator.generateConf(agent, new

Denny Ye