Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Flume NG - preferred alternative to TailSource


Copy link to this message
-
Re: Flume NG - preferred alternative to TailSource
Hi Jon,

         Flume-NG provides an exec source that executes a given
command and consumes the output. It should serve the purpose.

Regards,
    Mohammad Tariq
On Tue, Jul 17, 2012 at 10:52 PM, Jon Bender <[EMAIL PROTECTED]> wrote:
> Hi folks,
>
> I'm contemplating moving our current Flume 0.94 pipeline to Flume NG.  At
> the moment we make heavy use of the "multitail" source with some custom
> plugins to open up files/archives as they arrive in a specified directory,
> do some parsing and then emit the parsed events.
>
> Ideally I would replace this stage with some Flume NG counterpart.  Is there
> any Flume NG source that is designed to emit events from files, or is the
> suggestion that we build standalone parsers / tailers and transmit/receive
> the events over some standard RPC (using Avro, for instance)?
>
> Best,
> Jon