Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> about flume-ng agent


Copy link to this message
-
Re: about flume-ng agent
Hi,

Questions inline:

On Thu, Oct 25, 2012 at 11:54 AM, Paul Chavez
<[EMAIL PROTECTED]> wrote:
> I have tried to go the syslogUDP route to get log files from a Windows server to a flume agent, and did not find it an adequate solution.
>
> - We are seeing corruped events when sending IIS logs (known issue: https://issues.apache.org/jira/browse/FLUME-1365)
> - Our data is too large to fit in a 1500 byte ethernet frame so events are fragmented and the syslogUDP source ignores the continuation packets, resulting in truncated events.

I am not a Syslog expert, is there a way to detect the continuation
packets? If so I think we should open a JIRA on this issue.
> I have just been able to build the flume-ng agent for Windows and am testing the avro-client functionality on Windows. I think this will be the best bet for us in the short term, using LogParser to incrementally create files via scheduled task for the avro client to send along.
>
> Long term we want to develop a .net Avro client library for our apps to use directly. I suppose a log4net avro appender would be nice too.

I think a .NET version of the RPCClient and log4net appender would be
awesome.  If you do write them, would you considering pushing them
back to the flume community?

Brock