Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Chukwa >> mail # user >> multiple threads/HttpConnector from ChukwaAgent


Copy link to this message
-
Re: multiple threads/HttpConnector from ChukwaAgent
Hi Corbin,

This is by design.  We are concatenating all data streams into in
memory queue on the agent, and establish only one http connection to
collector.  This is for horizontal scalability that we can support
more machines.  At the same time, it also ensures that agent can write
more data per HTTP post to reduce overhead of HTTP headers and
connection handshakes.

regards,
Eric

On Thu, Jan 26, 2012 at 11:51 AM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:
> I am trying to do some real-time processing of the data coming into my chukwa pipeline and notice that using a single agent I don't seem to be getting very many servlets handling the requests. Peeking at the ChukwaAgent code it looks like the agents are limited to a single HttpConnector.
>
> Is this by design or am I off-base in my analysis of how it works?
>
>