Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> [jira] [Commented] (KAFKA-1124) Sending to a new topic (with auto.create.topics.enable) returns ERROR


+
Jason Rosenberg 2013-11-14, 14:49
Copy link to this message
-
[jira] [Commented] (KAFKA-1124) Sending to a new topic (with auto.create.topics.enable) returns ERROR

    [ https://issues.apache.org/jira/browse/KAFKA-1124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=13822177#comment-13822177 ]

Anuj Mehta commented on KAFKA-1124:
-----------------------------------

Hi

I tried reproducing this issue but I am getting a WARN message instead of an ERROR

[2013-11-13 18:24:59,310] WARN Error while fetching metadata [{TopicMetadata for topic newTopic ->
No partition metadata for topic newTopic due to kafka.common.LeaderNotAvailableException}] for topic [newTopic]: class kafka.common.LeaderNotAvailableException  (kafka.producer.BrokerPartitionInfo)

> Sending to a new topic (with auto.create.topics.enable) returns ERROR
> ---------------------------------------------------------------------
>
>                 Key: KAFKA-1124
>                 URL: https://issues.apache.org/jira/browse/KAFKA-1124
>             Project: Kafka
>          Issue Type: Bug
>    Affects Versions: 0.8
>            Reporter: Jason Rosenberg
>
> I had thought this was reported issue, but can't seem to find a previous report for it.
> If auto.create.topics.enable is true, a producer still gets an ERROR logged on the first attempt to send a message to a new topic, e.g.:
> 2013-11-06 03:00:08,638 ERROR [Thread-1] async.DefaultEventHandler - Failed to collate messages by topic, partition due to: Failed to fetch topic metadata for topic: mynewtopic
> 2013-11-06 03:00:08,638  INFO [Thread-1] async.DefaultEventHandler - Back off for 100 ms before retrying send. Remaining retries = 3
> This usually clears itself up immediately on retry (after 100 ms), as handled by the the kafka.producer.async.DefaultEventHandler (with retries enabled).
> However, this is logged to the client as an ERROR, and looks scary, when in fact it should have been a normal operation (since we have auto.create.topics.enable=true).
> There should be a better interaction here between the producer client and the server.
> Perhaps the server can create the topic in flight before returning the metadata request.
> Or, if it needs to be asynchronous, it could return a code which indicates something like: "The topic doesn't exist yet, it is being created, try again shortly".....and have the client automatically retry (even if retries not enabled, since it's not an ERROR condition, really).
> The ERROR log level is a problem since apps often have alert systems set up to notify when any ERROR happens, etc.....

--
This message was sent by Atlassian JIRA
(v6.1#6144)

 
+
Anuj Mehta 2013-11-19, 12:01
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB