Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # user >> How to use Esper in Flume


Copy link to this message
-
Re: How to use Esper in Flume
perhaps you can contribute this back into open source.
On Wed, Jun 5, 2013 at 8:02 AM, Jintao Guan <[EMAIL PROTECTED]> wrote:

> Thank you guys.
> Now I have written my custom sink of using Esper in Flume.
>
>
> On Tue, Jun 4, 2013 at 2:49 PM, Jintao Guan <[EMAIL PROTECTED]> wrote:
>
>> Hi Shekhar,
>>
>> Thank you for your email. The first solution is what I am looking for.
>> I really want to know the details of the custom sink.
>>
>> 1. The first thing I want to know is that how to install the custom sink
>> ? Compile it then put the ***.jar into $FLOME_HOME/lib ?
>> 2. Is there any material about how to write custom sink ?
>>
>> Thank you.
>>
>>
>> On Tue, Jun 4, 2013 at 2:41 PM, Shekhar Sharma <[EMAIL PROTECTED]>wrote:
>>
>>> You have two options :
>>>
>>> (1) You can write a custom flume sink for esper..Disadvantage would be
>>> you cant change the esper queries at run time
>>>
>>> (2) YOu can write esper service (Rest API) and the flume sink will send
>>> the java pojo to Esper service..The advantage is that you will have the
>>> ability to change the queries at run time..
>>>
>>> I have tried both the approaches..let me know if you would like to know
>>> more...
>>> The first approach is very simple, just need to extend the sink from
>>> flume Abstract sink and override the process method..
>>> In the constructor of the flume you need to initialze the esper engine,
>>> as well as esper statements and esper queries..
>>> Make sure your listeners are attached as well when constructing the
>>> object..
>>> While you receive the events in the process method, you need to convert
>>> the Flume SimpleEvent to POJO and call EsperEngine's sendevent() method...
>>>
>>> For the second method you can have your queries,listeners and statements
>>> defined in XML file and you parse them when the service starts and have a
>>> thread that looks for the change of this xml file..
>>>
>>> Hope that helps.
>>> Regards,
>>> Som Shekhar Sharma
>>> +91-8197243810
>>>
>>>
>>> On Tue, Jun 4, 2013 at 11:36 PM, Jintao Guan <[EMAIL PROTECTED]>wrote:
>>>
>>>> Hello guys,
>>>>
>>>> I want to use Esper in Flume Agents, so that I can monitor all of the
>>>> messages and logs and trigger alerts when there are Error logs.
>>>>
>>>>  Does it seem like a plugin ?  How do I install the custom interceptor ?
>>>>
>>>> Does anyone have any idea ?
>>>>
>>>
>>>
>>
>