Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Flume >> mail # dev >> NettyAvroRpcClient number of worker threads


+
Cameron Gandevia 2013-11-15, 23:41
+
Ashish 2013-11-16, 12:06
Copy link to this message
-
Re: NettyAvroRpcClient number of worker threads
Hi,

Awesome thanks for creating the ticket I will submit our patch against it.
My main concern was if other devs were ok exposing this setting to users

Thanks
On Nov 16, 2013 4:09 AM, "Ashish" <[EMAIL PROTECTED]> wrote:

> Created the JIRA https://issues.apache.org/jira/browse/FLUME-2238 and
> added
> credits to you.Please attach the patch to it.
>
> Solution would be to use threads as parameter to
> the NioServerSocketChannelFactory constructor using it as worker count or
> introduce a new variable for worker count.
> I would spend some time understanding the use of Worker Executor +
> NioWorker pool.
>
> thanks
> ashish
>
>
> On Sat, Nov 16, 2013 at 5:11 AM, Cameron Gandevia <[EMAIL PROTECTED]
> >wrote:
>
> > Hi,
> >
> > Recently we noticed some of our embedded flume agents were using around
> 50
> > netty IO worker threads.
> >
> > We tracked it down to the NettyAvroRpcClient which instantiates
> > the NioClientSocketChannelFactory with the default workerCount (2 * the
> > number of available processors). We would like finer control over the
> > number of IO threads we dedicate to flume.
> >
> > I can create a JIRA and a patch that allows users to configure this
> setting
> > on the AvroSink, but I wanted to bring it up with the flume developers
> > before proceeding.
> >
> > Thanks
> >
>
>
>
> --
> thanks
> ashish
>
> Blog: http://www.ashishpaliwal.com/blog
> My Photo Galleries: http://www.pbase.com/ashishpaliwal
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB