Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # dev - Event header validation using interceptors


Copy link to this message
-
Event header validation using interceptors
Nikolaos Tsipas 2014-02-13, 17:58
Hello,

We have a use case that requires the validation of headers on events received by an avro source in order to consider an event as valid or invalid. If an event is invalid then it should be routed to a different channel.

We know how to route events based on the values of specific headers using multiplexing. However, for the regex validation of headers flume doesn't seem to provide any appropriate interceptors.

For this reason, we are thinking to create a new interceptor that would allow regex validation of headers and depending on the outcome a header would be added (e.g. valid = true)

Questions:

* Does the above sound like a reasonable solution for what we want to achieve?
* What would be the best way to implement it in order to be beneficial for the flume community? Extend the functionality of one of the existing interceptors (e.g. RegexFilteringInterceptor) or provide a new one?

Regards,
Nikolaos
http://www.bbc.co.uk
This e-mail (and any attachments) is confidential and may contain personal views which are not the views of the BBC unless specifically stated.
If you have received it in error, please delete it from your system.
Do not use, copy or disclose the information in any way nor act in reliance on it and notify the sender immediately.
Please note that the BBC monitors e-mails sent or received.
Further communication will signify your consent to this.