Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # dev - layering


Copy link to this message
-
Re: layering
Chris Burroughs 2011-10-27, 00:03
This is great.  I know there are a lot of different use cases: embedded
vs standalone, some people want to avoid the ZK dependency etc.  This
will help us keep them all straight.

On 10/25/2011 12:54 AM, Jay Kreps wrote:
> Hey guys,
>
> I am seeing a few places where code is kind of growing together at the seams
> in a little. Here are two examples:
>
>    1. The kafka.log package is meant to be a stand-alone implementation of a
>    log. It shouldn't really be aware that it is inside a kafka server, or
>    connected to zookeeper or anything like that. But over time the zookeeper
>    registration has crept inside the LogManager so that no longer stands alone.
>    2. Likewise the network server is actually just a generic network server.
>    It is not supposed to know anything about kafka. But it has grown
>    dependencies to the kafka api objects.
>
> There are a few other minor ones like our cluster defintion (kafka.cluster)
> depending back on KafkaConfig.
>
> I made a script that draws the dependency diagram to help people visualize
> the current state, you can see it
>   https://issues.apache.org/jira/secure/attachment/12500611/kafka_deps.svg
>
> I filed a bug and will try to untangle a few of these once the new release
> is out the door:
> https://issues.apache.org/jira/browse/KAFKA-169
>
> Once we have cleaned it up a bit I think we can make a clearer statement
> about which systems depend on which other systems and add it to the coding
> guidelines.
>
> -Jay
>