Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> Clients and partition leaders


Copy link to this message
-
Re: Clients and partition leaders
This is a good idea. There are actually two ways to implement this:
1. A RESTFUL interface, as Jun mentions. This might make more sense
since if you don't mind the overhead of sending all the data twice
then you probably won't mind the overhead of HTTP.
2. Re-route misdirected requests in the brokers.

To effectively implement re-routing of requests requires a
non-blocking request router. Otherwise you end up blocking a thread
just waiting on the request. This might be okay (maybe just double the
number of threads) but isn't ideal. Our producer doesn't currently do
this kind of request pipelining, so a naive implementation that just
sent a produce request if the request wasn't local wouldn't quite do
it. The later strategy could be implemented much better when we have a
non-blocking producer.

-Jay
On Wed, Feb 13, 2013 at 7:08 AM, David Arthur <[EMAIL PROTECTED]> wrote:

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB