Kafka, mail # user - Re: Consumer throughput imbalance - 2013-08-26, 18:38
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Plain View
+
Ian Friedman 2013-08-24, 16:59
+
Neha Narkhede 2013-08-25, 14:46
+
Ian Friedman 2013-08-25, 16:01
+
Mark 2013-08-25, 16:14
+
Ian Friedman 2013-08-25, 17:17
+
Ian Friedman 2013-08-25, 17:24
+
Jay Kreps 2013-08-25, 19:12
+
Ian Friedman 2013-08-26, 05:02
+
Jay Kreps 2013-08-26, 17:27
+
Ian Friedman 2013-08-26, 18:19
+
Ian Friedman 2013-08-26, 18:34
Copy link to this message
-
Re: Consumer throughput imbalance
Yes exactly.

Lowering queuedchunks.max shouldn't help if the problem is what I
described. That options controls how many chunks the consumer has ready in
memory for processing. But we are hypothesisizing that your problem is
actually that the individual chunks are just too large leading to the
consumer spending a long time processing from one partition before it gets
the next chunk.

-Jay
On Mon, Aug 26, 2013 at 11:18 AM, Ian Friedman <[EMAIL PROTECTED]> wrote:
 
+
Ian Friedman 2013-08-26, 21:43
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB