Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> [jira] [Commented] (KAFKA-998) Producer should not retry on non-recoverable error codes


Copy link to this message
-
[jira] [Commented] (KAFKA-998) Producer should not retry on non-recoverable error codes

    [ https://issues.apache.org/jira/browse/KAFKA-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13758450#comment-13758450 ]

Joel Koshy commented on KAFKA-998:
----------------------------------

Apologies for the late review. Couple of comments:
* I think this could reset needRetry back to false if subsequent partitions in the iteration do need a retry: needRetry = needRetry && !fatalException(topicPartitionAndError._2). The logic is actually a bit confusing. Instead, it might be clearer to just do: failedTopicPartitions.exists(<some entry for which we need to retry>)
* Can you enhance the logging a bit to indicate that there were fatal sends that will not be retried? e.g., "Dropping messages to topic x due to message size limit.." or something like that.
* Can you rebase?

                
> Producer should not retry on non-recoverable error codes
> --------------------------------------------------------
>
>                 Key: KAFKA-998
>                 URL: https://issues.apache.org/jira/browse/KAFKA-998
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8, 0.8.1
>            Reporter: Joel Koshy
>            Assignee: Guozhang Wang
>         Attachments: KAFKA-998.v1.patch
>
>
> Based on a discussion with Guozhang. The producer currently retries on all error codes (including messagesizetoolarge which is pointless to retry on). This can slow down the producer unnecessarily.
> If at all we want to retry on that error code we would need to retry with a smaller batch size, but that's a separate discussion.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB