Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> doubt regarding the metadata.brokers.list parameter in producer properties


Copy link to this message
-
Re: doubt regarding the metadata.brokers.list parameter in producer properties
Since 0.8, broker discovery is not through zk anymore.

Guozhang
On Thu, Dec 19, 2013 at 8:45 AM, pushkar priyadarshi <
[EMAIL PROTECTED]> wrote:

> is the auto broker discovery not possible through zk.connect?
>
> Regards,
> Pushkar
>
>
> On Thu, Dec 19, 2013 at 9:29 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > metadata.broker.list is also used when there are leader failures. Could
> you
> > get a vip in ec2 and put the brokers' ip behind the vip?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Thu, Dec 19, 2013 at 4:07 AM, Arjun <[EMAIL PROTECTED]> wrote:
> >
> > > So from your reply what i understood is this particular property iis
> used
> > > only when starting the producers.
> > >
> > > is that right? can you please confirm.
> > >
> > > Thanks
> > > Arjun Narasimha Kota
> > >
> > >
> > > On Thursday 19 December 2013 05:33 PM, pushkar priyadarshi wrote:
> > >
> > >> 1.When you start producing : at this time if any of your supplied
> broker
> > >> is
> > >> alive system will continue to work.
> > >> 2.Broker going down and coming up with new IP : producer API refreshes
> > >> metadata information on failures(configurable) so they should be able
> to
> > >> detect new brokers.
> > >> But i dont think it's possible to ignore the initially supplied
> > parameter.
> > >>
> > >>
> > >> On Thu, Dec 19, 2013 at 4:57 PM, Arjun <[EMAIL PROTECTED]> wrote:
> > >>
> > >>  Hi,
> > >>>
> > >>> I am running kafka 0.8 and zoo keeper along with it. The problem i
> have
> > >>> is
> > >>> with the metadata.broker.list which is present in the producer
> > >>> properties.
> > >>> As i am using the zookeeper can i just ignore this property and kafka
> > >>> will
> > >>> work fine?
> > >>> We run kafka on the ec2 nodes, lets say we are running some 3 kafka
> > >>> servers, and producers metadata.broker.list populated with the ip's
> of
> > >>> these nodes, tomorrow if we add two more nodes for scaling up the
> > system
> > >>> we
> > >>> need to add those nodes to producers.
> > >>> Lets say in the above scenario(my nodes ip's are non elastic), then
> for
> > >>> some reason one of my nodes is down, and we brought up then the ip
> will
> > >>> get
> > >>> changed, so the producers should be changed to have new ip? what will
> > >>> happen if one after another all these servers are down and being
> > brought
> > >>> up?
> > >>>
> > >>> Is there any way I don't give the list but kafka with the help of
> > >>> zookeeper gets the list. I know kafka do not want to be dependent on
> > >>> zookeeper, but what should we do in such a case.
> > >>>
> > >>> Thanks
> > >>> Arjun Narasimha Kota
> > >>>
> > >>>
> > >
> >
>

--
-- Guozhang