Kafka, mail # user - Re: Time difference between message fetch and message send is very high (~900-1200ms) - 2013-05-17, 15:07
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Plain View
Kishore V. Kopalle 2013-05-17, 05:37
Kishore Kopalle 2013-05-17, 05:41
Francis Dallaire 2013-05-17, 06:09
Kishore V. Kopalle 2013-05-17, 06:12
Francis Dallaire 2013-05-17, 06:18
Kishore V. Kopalle 2013-05-17, 06:39
Neha Narkhede 2013-05-17, 13:38
Kishore V. Kopalle 2013-05-17, 13:57
Copy link to this message
Re: Time difference between message fetch and message send is very high (~900-1200ms)
There are mainly 2 things to consider for latency. (1) How quickly does the
producer send the message to the broker? If you use the sync mode in the
producer, the message will be sent immediately (throughput will be lower in
the sync mode though). If you use the async mode, the messages are sent
after enough messages have accumulated or enough time has passed. To reduce
latency, you need to reduce one or both thresholds. (2) How quickly does a
message get exposed to the consumer after it reaches the broker? In 0.7, a
message is exposed after it's flushed to disk.  By default, we flush the
log every 500 messages or 3 secs, for lower latency, you can try to reduce
log.flush.interval and default.log.flush.interval.ms. In 0.8, a message is
exposed to the consumer as soon as it reaches multiple brokers in memory.
So the latency will be better.


On Thu, May 16, 2013 at 10:37 PM, Kishore V. Kopalle <
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB