-Re: kafka internals -- and example client code
Philip O'Toole 2012-08-31, 16:35
Jay - thanks. And my understanding of the Scala docs is correct?
On Aug 30, 2012, at 2:52 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> Yeah, we definitely aren't talking to zk on every request. There is a hash
> map in memory that holds the active brokers, and that is updated when the
> zk watcher fires, which only happens when the set of brokers change.
> On Thu, Aug 30, 2012 at 9:33 AM, Philip O'Toole <[EMAIL PROTECTED]> wrote:
>> Yes -- thanks for this post.
>> I am new to Kafka, and I'd like clarification on one point. The
>> classes referenced by this post:
>> are the canonical Scala classes for writing Producer and Consumer
>> clients, correct? I am comparing these docs to the example clients
>> (particularly the Python and C++ examples). It seems the example
>> clients simply hard-code values such as "Partition ID", whereas these
>> docs show the complete way to access such information.
>> By the way, it seems that if one has to hit Zookeeper every time
>> before sending a message to Kafka, throughput will take a hit. If one
>> wants a high-performance system, clients must "use [a] local copy of
>> the list of brokers and their number of partitions". Is this also
>> Philip O'Toole
>> Senior Developer
>> Loggly, Inc.
>> San Francisco, CA
>> On Wed, Aug 29, 2012 at 6:12 PM, Pankaj Gupta <[EMAIL PROTECTED]>
>>> Hey Ming,
>>> Thanks for blogging. Kafka documentation is really good but it is always
>> good to see it from another perspective.
>>> On Aug 29, 2012, at 3:57 PM, Ming Han wrote:
>>>> I wrote a blog post about some of Kafka internals, if anyone is
>>>> Ming Han