I think encryption at the message level is a workable solution, as long as
you don't care about exposing the meta data that goes with it (e.g. topic
names, kafka broker/zk server locations, etc.).
On Tue, Apr 23, 2013 at 10:02 AM, Fergal Somers
> We are planning to use Kafka, but like others on this list we have a need
> to be able to secure communication. The approaches people have suggested on
> this list are:
> - Encrypt the messages at the producer (e.g
> - Add SSL to Kafka protocol -
> Adding SSL support to Kafka, probably means adding SSLEngine support the
> the nio socket handling (
> I don't think there are any immediate plans to provide this, but it's
> potentially something that Kafka would support in the future?
> In theory this is something we could look at, but we would need to go
> further. We also need to separate producers from consumers. The aim would
> be to ensure that a Kafka producer couldn't also act as a consumer.
> Essentially producers can write to Kafka, but not read.
> From looking at the Kafka source, achieving producer/consumer separation
> looks to me like it would be quite a change to the Kafka server (0.7). So
> are there any plans in the (near) future in this area (producer / consumer
> separation) ?
> Message encryption (at the application layer) would allow us to achieve
> both aims of securing communication and separating consumers from
> producers. Producers get the public cert (so they can encrypt messages as
> they place them on the bus). Only consumers get the private cert - so only
> they can decrypt messages consumed. This seems like something we can do
> ourselves - I just wanted to sanity check the approach with this group.