Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Parameters in Configuration File


Copy link to this message
-
Re: Parameters in Configuration File
I understand you have to continue to support existing clients. However, I don't know that total backwards compatibility would be desireable in the long run. If you want to support both the existing property configuration and a new one, one way would be to expose setters for each attribute of selectors, sinks, etc. This would let them be set directly, and allow the configuration to be moved to a separate factory class which would be used by the existing property files.

I have not dealt in detail with the configuration, though, so I'm not sure if there's something I'm overlooking.
From: Mike Percy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Fri, 15 Mar 2013 11:37:42 -0700
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Parameters in Configuration File

Hi Matt,
Properties files were the simple way to get started, and now that Flume is on version 1.3.1 we can't just abandon the properties file format.

I'm open to suggestions for adopting more full featured configuration functionality while maintaining backwards compatibility with the existing config files.

Regards,
Mike
On Fri, Mar 15, 2013 at 10:55 AM, Kenison, Matt <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
I'm sure this discussion has happened already, but is there a reason Spring (or something similar) isn't used for configuration, rather than the property files? It seems like there is a lot of boilerplate logic in the various implementations of Configurable.configure(Context) that is just completely unnecessary. In addition, you wouldn't have to spend time building something like property substitution that's been done many times before.
From: Mike Percy <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Reply-To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Date: Fri, 15 Mar 2013 10:27:00 -0700
To: "[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>" <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>>
Subject: Re: Parameters in Configuration File

Hey Connor,
Take a look at the discussion @ https://issues.apache.org/jira/browse/FLUME-1941

I you want to help work on this you are more than welcome to :)

Regards,
Mike

On Thu, Mar 14, 2013 at 5:09 PM, Connor Woodson <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
Does the Flume configuration file support parameters, for instance in the way that you can specify them in a log4j conf file?

- Connor
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB