Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Hardware Loadbalancer vs zookeeper for producers


Copy link to this message
-
Re: Hardware Loadbalancer vs zookeeper for producers
Take a look at the "Producer Load balancing" section of the documentation -
http://kafka.apache.org/08/documentation.html#theproducer

Let us know how we can improve it to answer your question.

Thanks,
Neha
On Wed, Sep 11, 2013 at 4:45 PM, Xuyen On <[EMAIL PROTECTED]> wrote:

> Hi Neha,
>
> How do you get metadata about the cluster from the hardware load balancer?
> A load balancer should direct a message to brokers that are up but I don't
> know how you would detect for things like adding/deleting topics/partitions
> from the cluster through the hardware load balancer. Can you help me
> understand how you guys do this?
>
> Thanks,
>
> Xuyen
>
> -----Original Message-----
> From: Neha Narkhede [mailto:[EMAIL PROTECTED]]
> Sent: Wednesday, September 11, 2013 2:39 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Hardware Loadbalancer vs zookeeper for producers
>
> Kafka 08 doesn't use zookeeper based load balancing on the producer side.
> We rely on a hardware load balancer to be able to talk to any broker and
> get metadata about the cluster. This happens on startup, for new
> topics/partitions and if the producer runs into an error while sending data.
>
> Thanks,
> Neha
>
>
> On Wed, Sep 11, 2013 at 12:38 PM, Xuyen On <[EMAIL PROTECTED]> wrote:
>
> > Can anyone from LinkedIn let me know if they are using Hardware
> > Loadbalancer to distribute the producer messages or if they're using
> > zookeeper?
> > I'm trying to find out what the pros and cons are of using a hardware
> > loadbalancer vs. using zookeeper to distribute the load of producers
> > sending messages to brokers. Does anyone have any experience using
> > either/both?
> >
> > Thanks,
> >
> > Xuyen
> >
> >
>
>
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB