Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume, mail # dev - NettyAvroRpcClient number of worker threads


+
Cameron Gandevia 2013-11-15, 23:41
+
Ashish 2013-11-16, 12:06
Copy link to this message
-
Re: NettyAvroRpcClient number of worker threads
Cameron Gandevia 2013-11-16, 19:13
Hi,

Awesome thanks for creating the ticket I will submit our patch against it.
My main concern was if other devs were ok exposing this setting to users

Thanks
On Nov 16, 2013 4:09 AM, "Ashish" <[EMAIL PROTECTED]> wrote:

> Created the JIRA https://issues.apache.org/jira/browse/FLUME-2238 and
> added
> credits to you.Please attach the patch to it.
>
> Solution would be to use threads as parameter to
> the NioServerSocketChannelFactory constructor using it as worker count or
> introduce a new variable for worker count.
> I would spend some time understanding the use of Worker Executor +
> NioWorker pool.
>
> thanks
> ashish
>
>
> On Sat, Nov 16, 2013 at 5:11 AM, Cameron Gandevia <[EMAIL PROTECTED]
> >wrote:
>
> > Hi,
> >
> > Recently we noticed some of our embedded flume agents were using around
> 50
> > netty IO worker threads.
> >
> > We tracked it down to the NettyAvroRpcClient which instantiates
> > the NioClientSocketChannelFactory with the default workerCount (2 * the
> > number of available processors). We would like finer control over the
> > number of IO threads we dedicate to flume.
> >
> > I can create a JIRA and a patch that allows users to configure this
> setting
> > on the AvroSink, but I wanted to bring it up with the flume developers
> > before proceeding.
> >
> > Thanks
> >
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>