Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> 0.8 producer -- many questions


Copy link to this message
-
0.8 producer -- many questions
Hello all,

I am writing a producer client for the v0.8 protocol.  I have the wire
format encoding working, and much of the functionality works well.
 However, I have made too many assumptions along the way, and it's time
that I figure out the answers to these things.  Sorry for the 'laundry
list' -- most of them are related, and can possibly be answered in batch
form.

TopicMetadata updates

   - is there a particular expectation of polling frequency?  (per topic,
   per broker, etc.?)
   - should a particular topic's metadata request only go to the "current"
   leader of that topic?
   - if such a request fails to receive response, is there a particular
   expected behavior?
   - I am assuming that producers always push to the leader (feels like
   it's in the docs someplace, but can't confirm)
   - if the leader changes, what is the protocol for a producer?  must the
   producer await ACK from outstanding requests before making a change, and if
   so, presumeably re-push those requests?

I'm sure there will be more, but this opens the door.

Cheers,

ben
+
Jun Rao 2012-11-26, 05:30
+
Jason Rosenberg 2012-11-26, 06:53
+
ben fleis 2012-11-26, 11:20
+
Joel Koshy 2012-11-27, 00:32
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB