Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> [jira] [Resolved] (KAFKA-1115) producer performance affected by trace/debug calls


Copy link to this message
-
[jira] [Resolved] (KAFKA-1115) producer performance affected by trace/debug calls

     [ https://issues.apache.org/jira/browse/KAFKA-1115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]

Joel Koshy resolved KAFKA-1115.
-------------------------------

    Resolution: Incomplete

So this is a non-issue right? Please re-open if you have any other concerns.

A lot of the trace/debug messages are often extremely important for live debugging. i.e., we set the level to debug/trace to avoid log flooding but when we do need to trouble-shoot a production issue we can enable those messages using the JMX operations exposed by kafka.utils.Log4jController

However if you see any log statements that can be improved/cleaned-up please let us know.

> producer performance affected by trace/debug calls
> --------------------------------------------------
>
>                 Key: KAFKA-1115
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1115
>             Project: Kafka
>          Issue Type: Improvement
>          Components: producer
>    Affects Versions: 0.8
>            Reporter: Francois Saint-Jacques
>            Assignee: Francois Saint-Jacques
>         Attachments: producer-performance-fix.patch
>
>
> After investigating high CPU usage on some producers in production, we found out that a lot of time was passed in string construction for logging of DEBUG and TRACE level.
> This patch removes most of the logging calls, on our systems it cuts CPU usage down to half of what it used to be.
> Note that this is a significant boost in performance for environment where there are thousands of msg/s.

--
This message was sent by Atlassian JIRA
(v6.1#6144)