Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Kafka 0.8 cluster setup?


+
Jason Huang 2013-04-22, 21:13
+
Neha Narkhede 2013-04-22, 21:32
+
Chris Curtin 2013-04-23, 12:29
+
Jason Huang 2013-04-23, 13:51
+
Eric Sites 2013-04-23, 14:22
Copy link to this message
-
Re: Kafka 0.8 cluster setup?
Jason Huang 2013-04-23, 14:31
Thanks Eric - this helps quite a bit.

I will play around with it.

Jason

On Tue, Apr 23, 2013 at 10:21 AM, Eric Sites <[EMAIL PROTECTED]>wrote:

> Jason,
>
> You need to modify the ZooKeeper config and add the following:
>
> dataDir=/tmp/zookeeper
>
> initLimit=50
> syncLimit=2
>
> server.1=kafka001.domain.com:2888:3888
> server.2=kafka002.domain.com:2888:3888
>
> server.3=kafka003.domain.com:2888:3888
>
>
>
> # Make sure you open those 2 points on each of the servers for your
> firewall
>
> The .1 after the server is the id of the zookeeper which should be stored
> in a file
> Named /tmp/zookeeper/myid
>
> Echo "1" > /tmp/zookeeper/myid`
>
>
> Cheers,
> Eric Sites
>
>
> On 4/23/13 9:50 AM, "Jason Huang" <[EMAIL PROTECTED]> wrote:
>
> >Thanks Chris and Neha.
> >
> >Chris - I've been through the link you mentioned before. However, that
> >appears to be using one instance of zookeeper, which makes whichever
> >server
> >that runs zookeeper as the single point of failure?
> >
> >Jason
> >
> >On Tue, Apr 23, 2013 at 8:28 AM, Chris Curtin
> ><[EMAIL PROTECTED]>wrote:
> >
> >> I following these instructions to get the first 'play' cluster going:
> >>
> >> https://cwiki.apache.org/KAFKA/kafka-08-quick-start.html
> >>
> >> Instead of running the 3 brokers on the same machine, I ran on on each
> >> machine.
> >>
> >> Note that you will need to do a little bit of zookeeper setup to get a
> >> cluster running, I think the main one was setting the 'myid' in the
> >> zookeeper install directory properly on each machine (namely give each
> >> machine a unique # in that file).
> >>
> >> Thanks,
> >>
> >> Chris
> >>
> >>
> >> On Mon, Apr 22, 2013 at 5:32 PM, Neha Narkhede <[EMAIL PROTECTED]
> >> >wrote:
> >>
> >> > Hi Jason,
> >> >
> >> > We are in the process of updating the documentation. Hoping to finish
> >> > it by this week. Stay tuned.
> >> >
> >> > Thanks,
> >> > Neha
> >> >
> >> > On Mon, Apr 22, 2013 at 2:12 PM, Jason Huang <[EMAIL PROTECTED]>
> >> > wrote:
> >> > > Hello,
> >> > >
> >> > > We've been playing around with kafka 0.8 for a few months now and
> >> decided
> >> > > to install kafka on a small cluster for further testing. I tried to
> >> > search
> >> > > online but couldn't find any setup documentation for a kafka 0.8
> >> cluster.
> >> > >
> >> > > Does anyone know if such documents exist? If they don't exist, what
> >> could
> >> > > be our best guide to start with?
> >> > >
> >> > > thanks!
> >> > >
> >> > > Jason
> >> >
> >>
>
>

 
+
Chris Curtin 2013-04-23, 14:42
+
Jason Huang 2013-04-23, 14:49
+
Jun Rao 2013-04-23, 15:43