Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Flume >> mail # dev >> NettyAvroRpcClient number of worker threads


Copy link to this message
-
Re: NettyAvroRpcClient number of worker threads
Created the JIRA https://issues.apache.org/jira/browse/FLUME-2238 and added
credits to you.Please attach the patch to it.

Solution would be to use threads as parameter to
the NioServerSocketChannelFactory constructor using it as worker count or
introduce a new variable for worker count.
I would spend some time understanding the use of Worker Executor +
NioWorker pool.

thanks
ashish
On Sat, Nov 16, 2013 at 5:11 AM, Cameron Gandevia <[EMAIL PROTECTED]>wrote:

> Hi,
>
> Recently we noticed some of our embedded flume agents were using around 50
> netty IO worker threads.
>
> We tracked it down to the NettyAvroRpcClient which instantiates
> the NioClientSocketChannelFactory with the default workerCount (2 * the
> number of available processors). We would like finer control over the
> number of IO threads we dedicate to flume.
>
> I can create a JIRA and a patch that allows users to configure this setting
> on the AvroSink, but I wanted to bring it up with the flume developers
> before proceeding.
>
> Thanks
>

--
thanks
ashish

Blog: http://www.ashishpaliwal.com/blog
My Photo Galleries: http://www.pbase.com/ashishpaliwal
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB