Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> Review Request: FLUME-1972 ElasticSearchSink now allows the id of the document to be provided by a custom class


Copy link to this message
-
Re: Review Request: FLUME-1972 ElasticSearchSink now allows the id of the document to be provided by a custom class

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/10380/#review18963
-----------------------------------------------------------

flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSink.java
<https://reviews.apache.org/r/10380/#comment39423>

    Sometimes, typos do occur in the configuration files for these custom class names.
    
    I think it would be nice to see if clazz is not null and also assignable before we try to instantiate an object for the provider.
    
    You should also log that the code was not able to create an instance from class specified in the config file.
    
    This will make things easier to troubleshoot when things go wrong.
    
    Take a look at an example here for some tips:
    
    org.apache.flume.serialization.EventSerializerFactory
- Israel Ekpo
On April 10, 2013, 5 p.m., Edward Sargisson wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/10380/
> -----------------------------------------------------------
>
> (Updated April 10, 2013, 5 p.m.)
>
>
> Review request for Flume and Hari Shreedharan.
>
>
> Description
> -------
>
> Elasticsearch allows clients to specify the id of the newly added document - and will generate one by default if none is supplied. If the user knows that the incoming event has a unique id then that id can be used as the elasticsearch document id. This change enhances the ElasticSearchSink to allow users to provide a custom class to decide what the id should be.
>
>
> This addresses bug FLUME-1972.
>     https://issues.apache.org/jira/browse/FLUME-1972
>
>
> Diffs
> -----
>
>   flume-ng-doc/sphinx/FlumeUserGuide.rst 693c0d7
>   flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchIdProvider.java PRE-CREATION
>   flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchNullIdProvider.java PRE-CREATION
>   flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSink.java 1b3db14
>   flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/ElasticSearchSinkConstants.java 7f75e22
>   flume-ng-sinks/flume-ng-elasticsearch-sink/src/main/java/org/apache/flume/sink/elasticsearch/IdHeaderElasticSearchIdProvider.java PRE-CREATION
>   flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/AbstractElasticSearchSinkTest.java 2edacdc
>   flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/IdHeaderElasticSearchIdProviderTest.java PRE-CREATION
>   flume-ng-sinks/flume-ng-elasticsearch-sink/src/test/java/org/apache/flume/sink/elasticsearch/TestElasticSearchSink.java 94b95b1
>
> Diff: https://reviews.apache.org/r/10380/diff/
>
>
> Testing
> -------
>
> All unit tests and integration tests pass. A snapshot using commit 5b9d31f1ad228 and the patch for flume-1972 has passed our internal integration tests using customisations.
>
>
> Thanks,
>
> Edward Sargisson
>
>