Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> [jira] [Resolved] (KAFKA-1115) producer performance affected by trace/debug calls


Copy link to this message
-
[jira] [Resolved] (KAFKA-1115) producer performance affected by trace/debug calls

     [ https://issues.apache.org/jira/browse/KAFKA-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joel Koshy resolved KAFKA-1115.
-------------------------------

    Resolution: Incomplete

So this is a non-issue right? Please re-open if you have any other concerns.

A lot of the trace/debug messages are often extremely important for live debugging. i.e., we set the level to debug/trace to avoid log flooding but when we do need to trouble-shoot a production issue we can enable those messages using the JMX operations exposed by kafka.utils.Log4jController

However if you see any log statements that can be improved/cleaned-up please let us know.

> producer performance affected by trace/debug calls
> --------------------------------------------------
>
>                 Key: KAFKA-1115
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1115
>             Project: Kafka
>          Issue Type: Improvement
>          Components: producer
>    Affects Versions: 0.8
>            Reporter: Francois Saint-Jacques
>            Assignee: Francois Saint-Jacques
>         Attachments: producer-performance-fix.patch
>
>
> After investigating high CPU usage on some producers in production, we found out that a lot of time was passed in string construction for logging of DEBUG and TRACE level.
> This patch removes most of the logging calls, on our systems it cuts CPU usage down to half of what it used to be.
> Note that this is a significant boost in performance for environment where there are thousands of msg/s.

--
This message was sent by Atlassian JIRA
(v6.1#6144)

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB