Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # user - Syslog-ng, Flume (old gen) and metadata


Copy link to this message
-
Syslog-ng, Flume (old gen) and metadata
Bertrand Dechoux 2012-10-29, 15:51
Hi,

I would like to deepen my understanding of syslog-ng and flume integration.

I had initial three questions :
1) Does flume timestamp is the same as the received syslog-ng event
timestamp?
2) Are the syslog-ng metadata kept by flume as extra metadata (> syslog-ng
OSE 3.0)?
3) Are the extra metadata (and potientially syslog-ng ones) usable with the
collector sink template?

>From what I understand :
1) Yes, acoording the cookbook. "use the timestamp found in the original
data,"
http://archive.cloudera.com/cdh/3/flume/Cookbook/
2) No but I haven't found a clear answer about it.
3) No but I haven't found a clear answer about it.

Could someone confirm me that?
I would believe 2) and 3) be generally asked features for those integrating
existent syslog-ng system with Flume but maybe it isn't. Or maybe there are
third party libraries for that? What is the status of Flume-ng with regards
to the same questions?

Regards

Bertrand
+
Brock Noland 2012-10-30, 15:12