Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> Review Request: FLUME-2015 ElasticSearchSink: need access to IndexRequestBuilder instance during flume event processing


+
Tim Bacon 2013-04-29, 08:43
+
Tim Bacon 2013-05-03, 00:24
+
Mike Percy 2013-05-08, 20:05
+
Mike Percy 2013-05-08, 20:29
+
Tim Bacon 2013-05-09, 09:38
Copy link to this message
-
Re: Review Request: FLUME-2015 ElasticSearchSink: need access to IndexRequestBuilder instance during flume event processing

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10835/
-----------------------------------------------------------

(Updated May 10, 2013, 10:08 a.m.)
Review request for Flume and Israel Ekpo.
Changes
-------

Changes from prior diff: ElasticSearchIndexRequestBuilderFactory interface should extend Configurable / ConfigurableComponent
Description
-------

This change adds an ElasticSearchIndexRequestBuilderFactory interface to allow users of an ElasticSearchSink to have greater control over the actual indexing of events that arrive at the sink. It is intended to meet the needs of:
- my own non-Kibana/non-logging-event use case
- the specific "allow for indexing an id-field" needs of FLUME-1972
- the (UTC) determination of the index to write to per FLUME-1782

This patch is backwards-compatible and imposes no changes on existing users of the sink.
This addresses bug FLUME-2015.
    https://issues.apache.org/jira/browse/FLUME-2015
Diffs (updated)
-----

  flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/AbstractElasticSearchIndexRequestBuilderFactory.java PRE-CREATION
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchEventSerializer.java dc6a093
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchIndexRequestBuilderFactory.java PRE-CREATION
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSink.java 1b3db14
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/EventSerializerIndexRequestBuilderFactory.java PRE-CREATION
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/AbstractElasticSearchSinkTest.java 2edacdc
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchIndexRequestBuilderFactory.java PRE-CREATION
  flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchSink.java 94b95b1

Diff: https://reviews.apache.org/r/10835/diff/
Testing
-------

Unit tests added and run.
Patch applied and run successfully in my own flume test environment.
Thanks,

Tim Bacon

+
Mike Percy 2013-05-10, 20:19