Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Multiplexing to multiple JdbcChannel (Derby) + event header ?


Copy link to this message
-
RE: Multiplexing to multiple JdbcChannel (Derby) + event header ?
Alain B. 2013-01-23, 08:01

Ok, I got it thank you. I'll follow your advice and go for file channels instead.

W.r.t. the interceptor, I'll implement  a custom org.apache.flume.interceptor class.
Just one more question: interceptors are configured in the source configuration. But are they executed before the events are inserted in the source or when they get out of the source (before going to the channel). If it's the later case, then I suppose it's done before any selector does its job (multiplexing on the basis of a header in my specific case).

So is it :
log4jappender -> interceptor -> source -> selector -> channel
or
log4jappender -> source -> interceptor -> selector -> channel

Thanks,
Alain
________________________________
> Date: Tue, 22 Jan 2013 10:54:49 -0800
> Subject: Re: Multiplexing to multiple JdbcChannel (Derby) + event header ?
> From: [EMAIL PROTECTED]
> To: [EMAIL PROTECTED]
>
>
> On Tue, Jan 22, 2013 at 2:51 AM, Alain B.
> <[EMAIL PROTECTED]<mailto:[EMAIL PROTECTED]>> wrote:
> My question is: will these 2 channels store their events in separate
> derby DB by default or do I need to configure my 2 jdbc-channels with
> specific properties in order to get 2 embedded derby DB started ?
>
>
> By default they will use the same derby DB.
> You can specify the location with the following property.
>
> a1.channels.c1.driver.url =
> jdbc:derby:/var/run/flume-ng/channel1/.flume/jdbc-channel/db;create=true
>
> a1.channels.c2.driver.url =
> jdbc:derby:/var/run/flume-ng/channel2/.flume/jdbc-channel/db;create=true
>
> As Hari already mentioned we Highly recommend the file channel for
> durability over the jdbc channel.