Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Questions about failure recovery, load balancing and producer ack/async mode


Copy link to this message
-
Re: Questions about failure recovery, load balancing and producer ack/async mode
1,3 Take a look at request.required.acks in
http://kafka.apache.org/documentation.html#producerconfigs

2. The producer does random distribution by default. However, you can
provide a partitioning key and a partitioning function. For details on how
consumer load balancing works, see
http://research.microsoft.com/en-us/um/people/srikanth/netdb11/netdb11papers/netdb11-final12.pdf
Thanks,

Jun
On Wed, Oct 23, 2013 at 12:12 PM, Shafaq <[EMAIL PROTECTED]> wrote:

> Hi,
>
>    Following need more elaboration after reading kafka docs:
>
> 1- In a scenario during leader fails over, what happens to messages that
> are not committed to other followers and to the messages that producer keep
> in sending (in async mode) till new leader is elected. Can the producer
> buffer these messages till the failover gets completed (Do we see
> exceptions. Are these all lost?
>
> 2. Broker load distribution or to be exact partition load distribution is
> done via hashed key mod no of partition, is that correct?  How is it done
> at the consumer side where one thread per partition (for high-level
> consumer) How is the faliover handlled
>
> 3. How do we configure producer for wait for leader ack and wait for
> commited act which Jun  Rao talks about in his 2013 apachecon presentation
>
> --
> Kind Regards,
> Shafaq
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB