Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Expected behavior for clustered kafka setup


+
Karl Kirch 2013-04-23, 15:50
+
Karl Kirch 2013-04-23, 16:02
+
Neha Narkhede 2013-04-23, 18:16
+
Karl Kirch 2013-04-23, 18:25
+
Neha Narkhede 2013-04-23, 18:28
+
Karl Kirch 2013-04-23, 18:36
+
Karl Kirch 2013-04-23, 18:56
Copy link to this message
-
Re: Expected behavior for clustered kafka setup
You should bring up your Zookeeper instances first and then the Kafka
brokers.
On Tue, Apr 23, 2013 at 11:56 AM, Karl Kirch <[EMAIL PROTECTED]> wrote:

> Now to make things even more interesting. I restarted 2 and now it sees
> all 3 nodes.
> I think I've got some sort of weirdness happening with how I'm bringing
> everything online.
>
> I've tried starting up all the zookeeper instances and then each of the
> brokers. I've tried starting up all of the kafka brokers and then the
> zookeeper instances. I've also tried bringing up each machine (zookeeper
> and kafka) up individually. All seem to give inconsistent results.
>
> Is there a recommended way to bring nodes online?
>
> Karl
>
> On Apr 23, 2013, at 1:36 PM, Karl Kirch <[EMAIL PROTECTED]> wrote:
>
> > I've tried doing what was suggested (creating the topic folders and
> restarting) and I'm getting slightly different but still bad results.
> > Now it'll write to node 1 and 3 but not to 2.
> >
> > Karl
> >
> > On Apr 23, 2013, at 1:27 PM, Neha Narkhede <[EMAIL PROTECTED]>
> > wrote:
> >
> >> Can you see if you are running to
> >> https://issues.apache.org/jira/browse/KAFKA-278 ?
> >>
> >> Thanks,
> >> Neha
> >>
> >> On Tue, Apr 23, 2013 at 11:24 AM, Karl Kirch <[EMAIL PROTECTED]> wrote:
> >>> Zookeeper based.
> >>>
> >>> Karl
> >>>
> >>> On Apr 23, 2013, at 1:15 PM, Neha Narkhede <[EMAIL PROTECTED]>
> >>> wrote:
> >>>
> >>>> What kind of producer do you use ? Zookeeper based or broker list ?
> >>>>
> >>>> Thanks,
> >>>> Neha
> >>>>
> >>>> On Tue, Apr 23, 2013 at 9:02 AM, Karl Kirch <[EMAIL PROTECTED]>
> wrote:
> >>>>> For some extra info, this was a restart of the just zookeeper and
> kafka, our clients were all still online.
> >>>>>
> >>>>> Karl
> >>>>>
> >>>>> On Apr 23, 2013, at 10:50 AM, Karl Kirch <[EMAIL PROTECTED]>
> >>>>> wrote:
> >>>>>
> >>>>>> Our setup is a three node cluster, we have a Kafka 0.7.2 (with 10
> partitions per node) and Zookeeper 3.3.4 on each node. I'm seeing that only
> one broker is being used to actually send messages through. We're seeing
> this behavior after a restart and deletion of the kafka data
> (/tmp/kafka-logs). In the past its used all three and worked fine.
> >>>>>>
> >>>>>> Is this normal? (i.e. it takes some time to spread across the
> cluster) Or should it distribute stuff immediately?
> >>>>>>
> >>>>>> Karl
> >>>>>
> >>>
> >
>
>

 
+
Karl Kirch 2013-04-23, 19:14
+
Jun Rao 2013-04-24, 01:06