Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> layering


This is great.  I know there are a lot of different use cases: embedded
vs standalone, some people want to avoid the ZK dependency etc.  This
will help us keep them all straight.

On 10/25/2011 12:54 AM, Jay Kreps wrote:
> Hey guys,
>
> I am seeing a few places where code is kind of growing together at the seams
> in a little. Here are two examples:
>
>    1. The kafka.log package is meant to be a stand-alone implementation of a
>    log. It shouldn't really be aware that it is inside a kafka server, or
>    connected to zookeeper or anything like that. But over time the zookeeper
>    registration has crept inside the LogManager so that no longer stands alone.
>    2. Likewise the network server is actually just a generic network server.
>    It is not supposed to know anything about kafka. But it has grown
>    dependencies to the kafka api objects.
>
> There are a few other minor ones like our cluster defintion (kafka.cluster)
> depending back on KafkaConfig.
>
> I made a script that draws the dependency diagram to help people visualize
> the current state, you can see it
>   https://issues.apache.org/jira/secure/attachment/12500611/kafka_deps.svg
>
> I filed a bug and will try to untangle a few of these once the new release
> is out the door:
> https://issues.apache.org/jira/browse/KAFKA-169
>
> Once we have cleaned it up a bit I think we can make a clearer statement
> about which systems depend on which other systems and add it to the coding
> guidelines.
>
> -Jay
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB