Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - flume-ng syslogtcp issue


+
Young Kim 2012-07-10, 20:59
+
alo alt 2012-07-11, 06:22
+
Young Kim 2012-07-11, 18:12
+
Young Kim 2012-07-12, 05:38
Copy link to this message
-
Re: flume-ng syslogtcp issue
Jarek Jarcec Cecho 2012-07-12, 05:48
Hi Young,
we've significantly improved user guide recently - you can get updated version either in repository (trunk) or in release 1.2.0 that is currently under vote (not released yet). Example of file_roll usage is present there.

Jarcec

On Jul 12, 2012, at 7:38 AM, Young Kim wrote:

> After some debugging, I found out that the syslogTcp is working as intended. I didn't realize that debug no longer shows the resulting syslog messages on success. On that note, this may be a bit off-topic, but is there documentation showing how file_roll should be used to write to local disk?
>
> Thanks,
> Young
> On Wednesday, July 11, 2012 at 11:12 AM, Young Kim wrote:
>
>> Hey Alex,
>>
>> The IP is correct, but I've altered it to say localhost. As for the syslog tests, I've ran it verbatim, but it doesn't register. However, when I alter /tmp/foo to include two new lines, flume registers it as a partial event.
>>
>> It still seems odd to me that flume-ng isn't detecting the syslogtcp whereas flume-og is doing it perfectly. Is it possible that there is a bug in the way flume-ng is implementing it?
>>
>> Thanks,
>> Young
>>
>> On Tuesday, July 10, 2012 at 11:22 PM, alo alt wrote:
>>
>>> HI,
>>>
>>> try these for syslog tests:
>>> echo "<13>Jun 20 12:12:12 host foo[345]: a syslog message" > /tmp/foo
>>> nc -v FLUME SYSLOG AGENT 5140 < /tmp/foo
>>>
>>>> syslog-agent.sources.syslog.host = 172.17.1.92
>>>
>>> Means, will only listen only on this interface, is the IP correct?
>>>
>>> cheers,
>>> Alex
>>>
>>> On Jul 10, 2012, at 10:59 PM, Young Kim wrote:
>>>
>>>> Hey guys,
>>>>
>>>> I've been trying to use flume-ng (build in trunk) to read syslogs through tcp then write to a flat file, as seen in this configuration:
>>>>
>>>> # Source, sink, and channel name below
>>>> syslog-agent.channels = rmc
>>>> syslog-agent.sources = syslog
>>>> syslog-agent.sinks = flat
>>>>
>>>> # Configure channels
>>>> syslog-agent.channels.rmc.type = memory
>>>>
>>>> # Configure sources
>>>> syslog-agent.sources.syslog.type = syslogTcp
>>>> syslog-agent.sources.syslog.port = 5140
>>>> syslog-agent.sources.syslog.host = 172.17.1.92
>>>> syslog-agent.sources.syslog.channels = rmc
>>>>
>>>> # Configure sinks
>>>> syslog-agent.sinks.flat.type = FILE_ROLL
>>>> syslog-agent.sinks.flat.directory = /Users/ykim/Desktop/test
>>>> syslog-agent.sinks.flat.rollInterval = 0
>>>> syslog-agent.sinks.flat.channels = rmc
>>>>
>>>>
>>>> After a bit of wrangling with rsyslog configurations, it appeared as if rsyslog wasn't sending any packets. However, I decided to try out flume-og with a simple dump syslogTcp, and it correctly reads the syslog. At this point, I'm a bit lost and not sure what else to look at. Hopefully, someone on here can shed some light on what could be wrong.
>>>>
>>>> Thanks,
>>>> Young Kim
>>>
>>>
>>> --
>>> Alexander Alten-Lorenz
>>> http://mapredit.blogspot.com
>>> German Hadoop LinkedIn Group: http://goo.gl/N8pCF
>>
>

+
Juhani Connolly 2012-07-12, 06:15
+
Stern, Mark 2012-07-12, 06:21
+
Hari Shreedharan 2012-07-12, 06:27