On 1/31/13 3:30 PM, Marc Labbe wrote:https://cwiki.apache.org/confluence/display/KAFKA/Consumer+Client+Re-Design
are the current design docs (as far as I know).
From my understanding, the client redesign has not been finalized and
it still in-progress/todo.
With 0.8, you will not need to connect to ZooKeeper from the clients.
With KAFKA-657, offsets are centrally managed by the broker. Any broker
can handle these requests.
I am working updating my Python client: https://github.com/mumrah/kafka-python
, still a ways to go yet. The
biggest change (besides centralized offset management) is that each
topic+partition is owned by a specific broker (the leader). When
producing messages, you must send them to the correct leader. This
requires that clients maintain some state of what belongs where which is
a pain, but such is the cost of replication.