Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Expected behavior for clustered kafka setup


+
Karl Kirch 2013-04-23, 15:50
+
Karl Kirch 2013-04-23, 16:02
+
Neha Narkhede 2013-04-23, 18:16
+
Karl Kirch 2013-04-23, 18:25
+
Neha Narkhede 2013-04-23, 18:28
+
Karl Kirch 2013-04-23, 18:36
+
Karl Kirch 2013-04-23, 18:56
Copy link to this message
-
Re: Expected behavior for clustered kafka setup
You should bring up your Zookeeper instances first and then the Kafka
brokers.
On Tue, Apr 23, 2013 at 11:56 AM, Karl Kirch <[EMAIL PROTECTED]> wrote:

> Now to make things even more interesting. I restarted 2 and now it sees
> all 3 nodes.
> I think I've got some sort of weirdness happening with how I'm bringing
> everything online.
>
> I've tried starting up all the zookeeper instances and then each of the
> brokers. I've tried starting up all of the kafka brokers and then the
> zookeeper instances. I've also tried bringing up each machine (zookeeper
> and kafka) up individually. All seem to give inconsistent results.
>
> Is there a recommended way to bring nodes online?
>
> Karl
>
> On Apr 23, 2013, at 1:36 PM, Karl Kirch <[EMAIL PROTECTED]> wrote:
>
> > I've tried doing what was suggested (creating the topic folders and
> restarting) and I'm getting slightly different but still bad results.
> > Now it'll write to node 1 and 3 but not to 2.
> >
> > Karl
> >
> > On Apr 23, 2013, at 1:27 PM, Neha Narkhede <[EMAIL PROTECTED]>
> > wrote:
> >
> >> Can you see if you are running to
> >> https://issues.apache.org/jira/browse/KAFKA-278 ?
> >>
> >> Thanks,
> >> Neha
> >>
> >> On Tue, Apr 23, 2013 at 11:24 AM, Karl Kirch <[EMAIL PROTECTED]> wrote:
> >>> Zookeeper based.
> >>>
> >>> Karl
> >>>
> >>> On Apr 23, 2013, at 1:15 PM, Neha Narkhede <[EMAIL PROTECTED]>
> >>> wrote:
> >>>
> >>>> What kind of producer do you use ? Zookeeper based or broker list ?
> >>>>
> >>>> Thanks,
> >>>> Neha
> >>>>
> >>>> On Tue, Apr 23, 2013 at 9:02 AM, Karl Kirch <[EMAIL PROTECTED]>
> wrote:
> >>>>> For some extra info, this was a restart of the just zookeeper and
> kafka, our clients were all still online.
> >>>>>
> >>>>> Karl
> >>>>>
> >>>>> On Apr 23, 2013, at 10:50 AM, Karl Kirch <[EMAIL PROTECTED]>
> >>>>> wrote:
> >>>>>
> >>>>>> Our setup is a three node cluster, we have a Kafka 0.7.2 (with 10
> partitions per node) and Zookeeper 3.3.4 on each node. I'm seeing that only
> one broker is being used to actually send messages through. We're seeing
> this behavior after a restart and deletion of the kafka data
> (/tmp/kafka-logs). In the past its used all three and worked fine.
> >>>>>>
> >>>>>> Is this normal? (i.e. it takes some time to spread across the
> cluster) Or should it distribute stuff immediately?
> >>>>>>
> >>>>>> Karl
> >>>>>
> >>>
> >
>
>

 
+
Karl Kirch 2013-04-23, 19:14
+
Jun Rao 2013-04-24, 01:06
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB