Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume NG - preferred alternative to TailSource


Copy link to this message
-
Re: Flume NG - preferred alternative to TailSource
Hi Jon,

         Flume-NG provides an exec source that executes a given
command and consumes the output. It should serve the purpose.

Regards,
    Mohammad Tariq
On Tue, Jul 17, 2012 at 10:52 PM, Jon Bender <[EMAIL PROTECTED]> wrote:
> Hi folks,
>
> I'm contemplating moving our current Flume 0.94 pipeline to Flume NG.  At
> the moment we make heavy use of the "multitail" source with some custom
> plugins to open up files/archives as they arrive in a specified directory,
> do some parsing and then emit the parsed events.
>
> Ideally I would replace this stage with some Flume NG counterpart.  Is there
> any Flume NG source that is designed to emit events from files, or is the
> suggestion that we build standalone parsers / tailers and transmit/receive
> the events over some standard RPC (using Avro, for instance)?
>
> Best,
> Jon
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB