Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Configuring flume agents remotely


Copy link to this message
-
Re: Configuring flume agents remotely
Monitoring and configuration are two separate things here.
Flume is typically monitored using either ganglia or http/json.
Both methods are documented here:
http://flume.apache.org/FlumeUserGuide.html#monitoring

As for configuration management and changes a common way of handling this
would be to
use a configuration management tool such as chef, puppet or ansible.
The agent polls the conf file every 30 seconds and if changes are detected
than the necessary components will be dynamically reloaded.

Not sure why Kafka is being mentioned as that is an entirely separate
project which has nothing to do with flume.
On Fri, Jan 3, 2014 at 3:19 AM, yavuz gokirmak <[EMAIL PROTECTED]> wrote:

> Hi all,
>
> We need to monitor and configure flume agents from a centralized tool. In
> previous versions as I know flume-master is used for such kind of
> requirements. Flume-master does not exist in flume-ng project.
>
> What is the adviced way for monitoring and configuring flume agents
> remotely *
>
> Assume I don't have ssh access to agent machines and therefore I need to
> get summary information such as #of events processed, #of files read etc..
> from running agents continuously
>
> And change agent configurations dynamically&remotely.
>
> regards
>