Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Avro >> mail # user >> thread pool for Socket server?


Copy link to this message
-
Re: thread pool for Socket server?
I'd say please do contribute a thread pool implementation.  Even if the
eventual answer is Netty, I've definitely run into issue.

-- Philip

On Wed, Jun 8, 2011 at 7:39 AM, Doug Cutting <[EMAIL PROTECTED]> wrote:

> A thread pool might indeed provide an improvement.  However the Netty
> and Jetty-based servers are probably better starting points for reliable
> performance.
>
> Also, please use SaslSocketServer instead of SocketSerer, with the
> anonymous mechanism if no security is required.  I'd like to standardize
> on SASL for non-HTTP, high-performance Avro RPC.
>
> It would be great if someone could contribute a Netty-based
> implementation of Avro's SASL profile.  That would provide a
> high-performance client and server that can support authentication
> and/or encryption.
>
> Doug
>
> On 06/08/2011 03:58 AM, Yang wrote:
> > currently SocketServer basically does this:
> >
> >       while (true) {
> >           new Connection(channel.accept());
> >
> >       }
> >
> >
> >
> > so it faithfully accepts any incoming connection and creates a worker
> > thread processing the request.
> >
> > I think this is prone to DOS attacks, or ungraceful failure in the case
> > of heavy traffic.
> >
> > if we use a limited thread pool inside Connection() implementation, we
> > can reject connections if we are too heavily loaded.
> >
> > thought it's basically a trivial thing to implement, but considering
> > this is a very common use case, is it worthwhile to put
> > such an implementation into the avro source?
> >
> > Thanks
> > Yang
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB