Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # user >> how to implement a tail or tailDir of flume-ng on windows?


+
周梦想 2013-02-21, 03:37
+
dan young 2013-02-21, 05:01
+
周梦想 2013-02-21, 06:22
Copy link to this message
-
Re: how to implement a tail or tailDir of flume-ng on windows?
You'd want to just periodically stat the file to be tailed, checking for
change in last modified/size, and read the difference out of it. You
could always download the source for tail itself and see how it does it:
http://git.savannah.gnu.org/cgit/coreutils.git/tree/src/tail.c

If you're going to write this to feed data to flume you're better off
having it send data over thrift to flume so you can resend it on failures.

On 02/21/2013 12:37 PM, 周梦想 wrote:
> hello,
>
> there isn't tail or tailDir source of flume-ng.
> exec source can run tail command on linux.
> but there is not a tail command on windows. So I have to write some
> code to do the same work.
> I want to read a file and if there is new lines of a file, it sends
> the lines to flume-ng.
>
> some one give me some advice?
>
> Thanks,
> Andy
>
+
周梦想 2013-02-21, 07:23
+
周梦想 2013-02-21, 08:22
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB