Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> Syslog-ng, Flume (old gen) and metadata


Copy link to this message
-
Syslog-ng, Flume (old gen) and metadata
Hi,

I would like to deepen my understanding of syslog-ng and flume integration.

I had initial three questions :
1) Does flume timestamp is the same as the received syslog-ng event
timestamp?
2) Are the syslog-ng metadata kept by flume as extra metadata (> syslog-ng
OSE 3.0)?
3) Are the extra metadata (and potientially syslog-ng ones) usable with the
collector sink template?

>From what I understand :
1) Yes, acoording the cookbook. "use the timestamp found in the original
data,"
http://archive.cloudera.com/cdh/3/flume/Cookbook/
2) No but I haven't found a clear answer about it.
3) No but I haven't found a clear answer about it.

Could someone confirm me that?
I would believe 2) and 3) be generally asked features for those integrating
existent syslog-ng system with Flume but maybe it isn't. Or maybe there are
third party libraries for that? What is the status of Flume-ng with regards
to the same questions?

Regards

Bertrand
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB