Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> doubt regarding the metadata.brokers.list parameter in producer properties


Copy link to this message
-
Re: doubt regarding the metadata.brokers.list parameter in producer properties
Since 0.8, broker discovery is not through zk anymore.

Guozhang
On Thu, Dec 19, 2013 at 8:45 AM, pushkar priyadarshi <
[EMAIL PROTECTED]> wrote:

> is the auto broker discovery not possible through zk.connect?
>
> Regards,
> Pushkar
>
>
> On Thu, Dec 19, 2013 at 9:29 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > metadata.broker.list is also used when there are leader failures. Could
> you
> > get a vip in ec2 and put the brokers' ip behind the vip?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Dec 19, 2013 at 4:07 AM, Arjun <[EMAIL PROTECTED]> wrote:
> >
> > > So from your reply what i understood is this particular property iis
> used
> > > only when starting the producers.
> > >
> > > is that right? can you please confirm.
> > >
> > > Thanks
> > > Arjun Narasimha Kota
> > >
> > >
> > > On Thursday 19 December 2013 05:33 PM, pushkar priyadarshi wrote:
> > >
> > >> 1.When you start producing : at this time if any of your supplied
> broker
> > >> is
> > >> alive system will continue to work.
> > >> 2.Broker going down and coming up with new IP : producer API refreshes
> > >> metadata information on failures(configurable) so they should be able
> to
> > >> detect new brokers.
> > >> But i dont think it's possible to ignore the initially supplied
> > parameter.
> > >>
> > >>
> > >> On Thu, Dec 19, 2013 at 4:57 PM, Arjun <[EMAIL PROTECTED]> wrote:
> > >>
> > >>  Hi,
> > >>>
> > >>> I am running kafka 0.8 and zoo keeper along with it. The problem i
> have
> > >>> is
> > >>> with the metadata.broker.list which is present in the producer
> > >>> properties.
> > >>> As i am using the zookeeper can i just ignore this property and kafka
> > >>> will
> > >>> work fine?
> > >>> We run kafka on the ec2 nodes, lets say we are running some 3 kafka
> > >>> servers, and producers metadata.broker.list populated with the ip's
> of
> > >>> these nodes, tomorrow if we add two more nodes for scaling up the
> > system
> > >>> we
> > >>> need to add those nodes to producers.
> > >>> Lets say in the above scenario(my nodes ip's are non elastic), then
> for
> > >>> some reason one of my nodes is down, and we brought up then the ip
> will
> > >>> get
> > >>> changed, so the producers should be changed to have new ip? what will
> > >>> happen if one after another all these servers are down and being
> > brought
> > >>> up?
> > >>>
> > >>> Is there any way I don't give the list but kafka with the help of
> > >>> zookeeper gets the list. I know kafka do not want to be dependent on
> > >>> zookeeper, but what should we do in such a case.
> > >>>
> > >>> Thanks
> > >>> Arjun Narasimha Kota
> > >>>
> > >>>
> > >
> >
>

--
-- Guozhang

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB