Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request 14614: Change the way ElasticSearchSink accepts ttl field to accept TTL qualifiers


Copy link to this message
-
Review Request 14614: Change the way ElasticSearchSink accepts ttl field to accept TTL qualifiers

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/14614/
-----------------------------------------------------------

Review request for Flume.
Bugs: FLUME-2206
    https://issues.apache.org/jira/browse/FLUME-2206
Repository: flume-git
Description
-------

- Added parseTTL method to ElasticSearchSink to parse and map ttl specified by user to corresponding ttl value in milliseconds. parseTTL() defaults to days when no qualifier is specified by the user. Example: 1 and 1d will both return 86400000 which is 1 day in milliseconds.
- Added shouldParseFullyQualifiedTTLs() in TestElasticSearchSink class to run through a list of TTLs with various qualifiers.
- Updated FlumeUserGuide.rst
- Added TTL parsing regex to ElasticSearchSinkConstants
Diffs
-----

  .gitignore ef0a495
  flume-ng-doc/sphinx/FlumeUserGuide.rst 98859ce
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSink.java 3286412
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSinkConstants.java 7f75e22
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchSink.java ad40a3c

Diff: https://reviews.apache.org/r/14614/diff/
Testing
-------

- Built flume and tested in local development setup for flume. Further testing required to actually test if the TTL set has any effect on ElasticSearch index / document deletion.
Thanks,

Dib Ghosh