Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> [jira] [Commented] (KAFKA-998) Producer should not retry on non-recoverable error codes


Copy link to this message
-
[jira] [Commented] (KAFKA-998) Producer should not retry on non-recoverable error codes

    [ https://issues.apache.org/jira/browse/KAFKA-998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13729681#comment-13729681 ]

Guozhang Wang commented on KAFKA-998:
-------------------------------------

Approach proposal:

1. Passing the errorCode from send to dispatchSerializedData to handle along with the outstandingProduceRequests.

2. When outstandingProduceRequests.size > 0, check the corresponding error code, if the error code indicates a non-avoidable error such as MessageSizeTooLarge, then break the while loop intermediately.

By doing so it would also be easier in the future if we want to make a smarted producer client by dynamically shrinking the batch size.
                
> Producer should not retry on non-recoverable error codes
> --------------------------------------------------------
>
>                 Key: KAFKA-998
>                 URL: https://issues.apache.org/jira/browse/KAFKA-998
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8, 0.8.1
>            Reporter: Joel Koshy
>            Assignee: Guozhang Wang
>
> Based on a discussion with Guozhang. The producer currently retries on all error codes (including messagesizetoolarge which is pointless to retry on). This can slow down the producer unnecessarily.
> If at all we want to retry on that error code we would need to retry with a smaller batch size, but that's a separate discussion.

--
This message is automatically generated by JIRA.
If you think it was sent incorrectly, please contact your JIRA administrators
For more information on JIRA, see: http://www.atlassian.com/software/jira

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB