Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Chukwa >> mail # user >> multiple threads/HttpConnector from ChukwaAgent

Corbin Hoenes 2012-01-26, 19:51
Copy link to this message
Re: multiple threads/HttpConnector from ChukwaAgent
Hi Corbin,

This is by design.  We are concatenating all data streams into in
memory queue on the agent, and establish only one http connection to
collector.  This is for horizontal scalability that we can support
more machines.  At the same time, it also ensures that agent can write
more data per HTTP post to reduce overhead of HTTP headers and
connection handshakes.


On Thu, Jan 26, 2012 at 11:51 AM, Corbin Hoenes <[EMAIL PROTECTED]> wrote:
> I am trying to do some real-time processing of the data coming into my chukwa pipeline and notice that using a single agent I don't seem to be getting very many servlets handling the requests. Peeking at the ChukwaAgent code it looks like the agents are limited to a single HttpConnector.
> Is this by design or am I off-base in my analysis of how it works?
Corbin Hoenes 2012-01-26, 21:27
Eric Yang 2012-01-27, 02:02