Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - 0.8 producer -- many questions


Copy link to this message
-
0.8 producer -- many questions
ben fleis 2012-11-23, 15:51
Hello all,

I am writing a producer client for the v0.8 protocol.  I have the wire
format encoding working, and much of the functionality works well.
 However, I have made too many assumptions along the way, and it's time
that I figure out the answers to these things.  Sorry for the 'laundry
list' -- most of them are related, and can possibly be answered in batch
form.

TopicMetadata updates

   - is there a particular expectation of polling frequency?  (per topic,
   per broker, etc.?)
   - should a particular topic's metadata request only go to the "current"
   leader of that topic?
   - if such a request fails to receive response, is there a particular
   expected behavior?
   - I am assuming that producers always push to the leader (feels like
   it's in the docs someplace, but can't confirm)
   - if the leader changes, what is the protocol for a producer?  must the
   producer await ACK from outstanding requests before making a change, and if
   so, presumeably re-push those requests?

I'm sure there will be more, but this opens the door.

Cheers,

ben
+
Jun Rao 2012-11-26, 05:30
+
Jason Rosenberg 2012-11-26, 06:53
+
ben fleis 2012-11-26, 11:20
+
Joel Koshy 2012-11-27, 00:32