I have a 3 node kafka cluster, and i have 2 topics( one of very high priority and other normal data), i need to transmit the normal data to two brokers in the cluster , and the high priority data directly to the 3rd broker.
This is so that my high priority data has a clear path and can be transmitted without any delay at all.
What should I do to achieve this ? Just specifying appropriate broker.list for each producer , is enough? On Tue, May 14, 2013 at 9:11 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote: Chitra Raveendran Data Scientist *Flutura Business Solutions Pvt. Ltd* Tel : +918197563660 email : [EMAIL PROTECTED]
Do you have any tests that measure that your high priority data is being delayed ? Assuming you are using 0.8, the end to end latency can be reduced by tuning some configs on the consumer (fetch.min.bytes, fetch.wait.max.ms ). The defaults for these configs are already tuned for low latency though.
Thanks, Neha On Tue, May 14, 2013 at 11:46 AM, Chitra Raveendran < [EMAIL PROTECTED]> wrote:
The feature I mentioned is only available on 0.8. In 0.7.2, you can tweak producer batch size and the flush interval on the broker for the high priority topics. Note that setting those too low will have performance implications.
Thanks, Neha On May 17, 2013 2:25 PM, "Neha Narkhede" <[EMAIL PROTECTED]> wrote:
NEW: Monitor These Apps!
Apache Lucene, Apache Solr and all other Apache Software Foundation project and their respective logos are trademarks of the Apache Software Foundation.
Elasticsearch, Kibana, Logstash, and Beats are trademarks of Elasticsearch BV, registered in the U.S. and in other countries. This site and Sematext Group is in no way affiliated with Elasticsearch BV.
Service operated by Sematext