Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> Does Flume NG requires to be installed on all the sources?


+
Seshu V 2013-02-07, 00:49
Copy link to this message
-
Re: Does Flume NG requires to be installed on all the sources?
Seshu,

It really is going to depend on your use case.
Though it sounds that you may need to run an agent on each of the source
machines.
Which source do you plan to use? It may also be the case that you can use
the flume rpc client to write data directly from your application to the
flume collector machine.

http://flume.apache.org/FlumeDeveloperGuide.html#rpc-client-interface

-Jeff
On Wed, Feb 6, 2013 at 4:49 PM, Seshu V <[EMAIL PROTECTED]> wrote:

> Hi All,
>
>     I have used Flume 0.9.3 a while back, it worked fine at that time.
>  Now, I am looking to use 'Flume NG', started reading documentation today.
>  In Flume 0.9.3, I installed flume agents on the servers wherever I had the
> data source.   And, I had a collector machine separately.  My sink was
> HDFS.   I see that Flume NG is using Channel.
>     My question is that I have multiple source servers and my sink is
> HDFS.  I also have another machine for Channel (collector in old days).
> Do I need to install flume NG  in all the source machines and Channel
> machine?  Or can I install flume NG only on the Channel server and
> (somehow) specify in the configuration to pull data from source machines
> and specify the sink as HDFS?
>      Thanks in advance for your replies..
>
> Thanks,
> - Seshu
>
>
+
Seshu V 2013-02-07, 18:07
+
Jeff Lord 2013-02-08, 22:42
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB