Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Securing Kafka

Copy link to this message
Re: Securing Kafka

I think encryption at the message level is a workable solution, as long as
you don't care about exposing the meta data that goes with it (e.g. topic
names, kafka broker/zk server locations, etc.).

On Tue, Apr 23, 2013 at 10:02 AM, Fergal Somers

> Hi
> We are planning to use Kafka, but like others on this list we have a need
> to be able to secure communication. The approaches people have suggested on
> this list are:
>    - Encrypt the messages at the producer (e.g
> http://search-hadoop.com/m/1AfXKcZIk52/message+encryption&subj=Re+Secure+communication
>    )
>    - Add SSL to Kafka protocol -
> http://mail-archives.apache.org/mod_mbox/kafka-users/201304.mbox/ajax/%3CCAA%2BBczQ_dMXUTNndSu4d%2B6aRo%3DSLiFa4iGMu_78OWKub_CTScw%40mail.gmail.com%3E
> Adding SSL support to Kafka, probably means adding SSLEngine support the
> the nio socket handling (
> https://groups.google.com/forum/#!msg/kafka-dev/ZmJrB_plu1I/_9cmGlLCSVEJ).
> I don't think there are any immediate plans to provide this, but it's
> potentially something that Kafka would support in the future?
> In theory this is something we could look at, but we would need to go
> further. We also need to separate producers from consumers. The aim would
> be to ensure that a Kafka producer couldn't also act as a consumer.
> Essentially producers can write to Kafka, but not read.
> From looking at the Kafka source, achieving producer/consumer separation
> looks to me like it would be quite a change to the Kafka server (0.7). So
> are there any plans in the (near) future in this area (producer / consumer
> separation) ?
> Message encryption (at the application layer) would allow us to achieve
> both aims of securing communication and separating consumers from
> producers. Producers get the public cert (so they can encrypt messages as
> they place them on the bus). Only consumers get the private cert - so only
> they can decrypt messages consumed. This seems like something we can do
> ourselves - I just wanted to sanity check the approach with this group.
> Cheers,
> Fergal.