While the replication features in 0.8 are very desirable for us, one aspect of 0.7 that was also appealing was that in specific scenarios a single broker instance could run by itself without an accompanying Zookeeper.  

This provided a lightweight "entry point" for log flows by running lots of single-instance Kafka brokers for local durability, then running other code (like MirrorMaker) to propagate local data from those single-instance brokers to shared broker clusters for further processing by other applications.

Is this scenario no longer achievable with 0.8? Will it be on the roadmap again?
 

Phil,

In 0.8, the broker always depends on ZK. So, this is no longer optional.
You can still run a single node broker with replication factor set to 1
though.

Actually, in 0.7, MirrorMaker depends on ZK to consume data in a broker.
So, how did you get the data from the local broker out without ZK?

Thanks,

Jun

On Mon, Mar 18, 2013 at 6:52 AM, Hargett, Phil <
[EMAIL PROTECTED]> wrote:

> While the replication features in 0.8 are very desirable for us, one
> aspect of 0.7 that was also appealing was that in specific scenarios a
> single broker instance could run by itself without an accompanying
> Zookeeper.
>
> This provided a lightweight "entry point" for log flows by running lots of
> single-instance Kafka brokers for local durability, then running other code
> (like MirrorMaker) to propagate local data from those single-instance
> brokers to shared broker clusters for further processing by other
> applications.
>
> Is this scenario no longer achievable with 0.8? Will it be on the roadmap
> again?

 

Re "So, how did you get the data from the local broker out without ZK"...

We didn't use Mirror Maker itself.  We wrote a simple application inspired by Mirror Maker but written in Java that understands our topology and used external information to locate source brokers from which to consume data.

:)