Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> changing broker hosts with 0.7.2


+
Jason Rosenberg 2013-03-19, 21:04
+
Neha Narkhede 2013-03-20, 02:07
+
Jason Rosenberg 2013-03-20, 06:26
+
Neha Narkhede 2013-03-20, 13:42
+
Jason Rosenberg 2013-03-20, 15:56
+
Neha Narkhede 2013-03-20, 16:02
+
Philip OToole 2013-03-20, 16:07
+
Jason Rosenberg 2013-03-20, 17:55
+
Philip OToole 2013-03-20, 18:10
+
Jason Rosenberg 2013-03-20, 18:21
+
Philip OToole 2013-03-20, 19:00
+
Jason Rosenberg 2013-03-20, 19:07
+
Philip OToole 2013-03-20, 19:16
Copy link to this message
-
Re: changing broker hosts with 0.7.2
Ok,

I programmatically configure my kafka producers, but essentially configure
them passing a set of config properties, as if specified in a .properties
file.

I'll think about trying this, seems like it might just work.

Jason
On Wed, Mar 20, 2013 at 12:16 PM, Philip O'Toole <[EMAIL PROTECTED]> wrote:

> On Wed, Mar 20, 2013 at 12:06 PM, Jason Rosenberg <[EMAIL PROTECTED]>
> wrote:
> > On Wed, Mar 20, 2013 at 12:00 PM, Philip O'Toole <[EMAIL PROTECTED]>
> wrote:
> >
> >> > For
> >> > producers, also, you can't really use a load-balancer to connect to
> >> brokers
> >> > (you can use zk, or you can use a broker list, in 0.7.2, and in 0.8,
> you
> >> > can use an LB for the initial meta data connection, but then you still
> >> have
> >> > to have direct connections to each broker from each producer).
> >>
> >> Huh? Sure you can, if the Producers are simple. Producers just need a
> >> destination IP address and port. They have no way of knowing if that
> >> IP is an LB or real Kafka broker. Set the partition to -1 (i.e.
> >> random) in all messages destined for Kafka, and it all just works.
> >> Granted, this is a simple type of 0.72 deployment. Perhaps it doesn't
> >> work for more complex Producers or 0.8 deployments -- I am not yet
> >> that familiar with 0.8.
> >
> >
> > Interesting, so in this case, do you just provide the vip in the
> > "broker.list" property for the producer?  That might just work for my
> needs.
>
> We have custom non-Java Producers so don't have Property files, but in
> principle yes, that is what I mean.
>
> Philip
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB