Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> changing broker hosts with 0.7.2


+
Jason Rosenberg 2013-03-19, 21:04
+
Neha Narkhede 2013-03-20, 02:07
+
Jason Rosenberg 2013-03-20, 06:26
+
Neha Narkhede 2013-03-20, 13:42
+
Jason Rosenberg 2013-03-20, 15:56
+
Neha Narkhede 2013-03-20, 16:02
+
Philip OToole 2013-03-20, 16:07
+
Jason Rosenberg 2013-03-20, 17:55
+
Philip OToole 2013-03-20, 18:10
+
Jason Rosenberg 2013-03-20, 18:21
+
Philip OToole 2013-03-20, 19:00
+
Jason Rosenberg 2013-03-20, 19:07
+
Philip OToole 2013-03-20, 19:16
Copy link to this message
-
Re: changing broker hosts with 0.7.2
Ok,

I programmatically configure my kafka producers, but essentially configure
them passing a set of config properties, as if specified in a .properties
file.

I'll think about trying this, seems like it might just work.

Jason
On Wed, Mar 20, 2013 at 12:16 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:

> On Wed, Mar 20, 2013 at 12:06 PM, Jason Rosenberg <[EMAIL PROTECTED]>
> wrote:
> > On Wed, Mar 20, 2013 at 12:00 PM, Philip O'Toole <[EMAIL PROTECTED]>
> wrote:
> >
> >> > For
> >> > producers, also, you can't really use a load-balancer to connect to
> >> brokers
> >> > (you can use zk, or you can use a broker list, in 0.7.2, and in 0.8,
> you
> >> > can use an LB for the initial meta data connection, but then you still
> >> have
> >> > to have direct connections to each broker from each producer).
> >>
> >> Huh? Sure you can, if the Producers are simple. Producers just need a
> >> destination IP address and port. They have no way of knowing if that
> >> IP is an LB or real Kafka broker. Set the partition to -1 (i.e.
> >> random) in all messages destined for Kafka, and it all just works.
> >> Granted, this is a simple type of 0.72 deployment. Perhaps it doesn't
> >> work for more complex Producers or 0.8 deployments -- I am not yet
> >> that familiar with 0.8.
> >
> >
> > Interesting, so in this case, do you just provide the vip in the
> > "broker.list" property for the producer?  That might just work for my
> needs.
>
> We have custom non-Java Producers so don't have Property files, but in
> principle yes, that is what I mean.
>
> Philip
>