Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume, mail # user - Several sinks from the same channel


Copy link to this message
-
Re: Several sinks from the same channel
Hari Shreedharan 2012-09-17, 04:29
Yes, each sink will be run by a different thread called the SinkRunner. Several sinks are multithreaded internally too(like the HDFSEventSink/AsyncHBaseSink).  

Hari  

--  
Hari Shreedharan
On Sunday, September 16, 2012 at 4:24 PM, João Ascenso wrote:

> Thanks for the response
>  
> Very nice design on flume part for the sink :)
>  
> So, just to clarify, if i have 6 sinks running operations on logs before writing, each one will be on a diferent thread/ not hogging the same core on the cpu?  
> ----------------
> João Ascenso
>  
>  
> 2012/9/16 Hari Shreedharan <[EMAIL PROTECTED] (mailto:[EMAIL PROTECTED])>
> > Reply inline.  
> >  
> > --  
> > Hari Shreedharan
> >  
> >  
> > On Sunday, September 16, 2012 at 1:18 PM, João Ascenso wrote:
> >  
> > > Greetings,
> > >  
> > > Anyone knows if i put n sinks listening to one channel, if every sink is going to get copies of the message or if the messages are distributed between the sinks?  
> > Events will get distributed across multiple sinks. The channel does not really care if one sink or multiple sinks take the event. Once taken and committed, the channel deletes the event.  
> > >  
> > > Also, every new sink is gonna run on his own thread, or do i need several flume instances for that?
> > No, you can have multiple sinks in the same agent.  
> > >  
> > > -------------------
> > > João Ascenso
> >  
>