-Re: Events stored in Apache Web Logs - how best to get these into Kafka?
Jun Rao 2013-06-12, 05:27
As long as you can pipe log data to console-producer, you can get the data
On Tue, Jun 11, 2013 at 2:57 PM, James Newhaven <[EMAIL PROTECTED]>wrote:
> I have a lot of user event data being sent to an apache web server and
> written to web logs. Unfortunately I don't control this flow, but do I have
> access to the server the logs are being written to.
> I need a way to access these events as soon as they have been written to
> the log file and send this over to Kafka.
> I'm aware I can write a Kafka Producer, but I was hoping there might be
> something out of the box I could use that could monitor a log file and send
> the contents to Kafka.
> I saw from KAFKA-130 <https://issues.apache.org/jira/browse/KAFKA-130>
> in version 0.7 onwards there is something that can monitor STDIN, not sure
> if I could use this for web log files?