Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Running Kafka in local test mode

Ken Krugler 2012-05-06, 15:21
Ken Krugler 2012-05-06, 16:53
Jun Rao 2012-05-07, 15:10
Ken Krugler 2012-05-09, 00:34
Jun Rao 2012-05-09, 00:40
Ken Krugler 2012-05-09, 19:01
Jun Rao 2012-05-09, 21:03
Ken Krugler 2012-05-09, 21:47
Jun Rao 2012-05-10, 00:12
Hisham Mardam-Bey 2012-05-10, 00:20
Copy link to this message
Re: Running Kafka in local test mode
Hi Hisham,

Thanks for chiming in - see below.

> When you call shutdown() on KafkaServerStartable you should also call
> awaitShutdown if you're expecting your app / JVM to terminate (perhaps
> you are not).

It's been shutting down properly, but you're right that calling awaitShutdown() ensures that everything has been properly cleaned up before I continue.

> Also take into account that if the embedded KafkaServer
> instance in KafkaServerStartable throws an exception on shutdown it
> will call Runtime.getRuntime.halt(1) (not sure if you want this
> behaviour).

Not really, but hopefully my unit tests aren't triggering exceptions in the KafkaServer :)

> I'm embedding a Kafka broker and producer in an experimental lib doing
> pretty much what you're doing except I use KafkaServer directly and
> make no use of ZK.

I initially tried that, but ran into the issue where Kafka consumers always create a ZooKeeper client, which expects to have a ZooKeeper server running.

Do you know if it's possible to set up a complete Kafka environment (broker, producer, consumer) without ZooKeeper?

-- Ken

Ken Krugler
custom big data solutions & training
Hadoop, Cascading, Mahout & Solr
Hisham Mardam-Bey 2012-05-10, 03:00
Ken Krugler 2012-06-28, 03:19