Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> code layout


Copy link to this message
-
Re: code layout
That's why I was thinking c) makes sense from a design perspective.

> On Feb 2, 2014, at 21:59, Jay Kreps <[EMAIL PROTECTED]> wrote:
>
> Yes, this is a good point, the admin api could depend on zookeeper. It does
> today. That would not work in the existing clients package.
>
> -Jay
>
>
>> On Sun, Feb 2, 2014 at 4:20 PM, Steve Morin <[EMAIL PROTECTED]> wrote:
>>
>> Would the admin api's potentially have different dependencies?
>>
>>> On Feb 1, 2014, at 15:06, Neha Narkhede <[EMAIL PROTECTED]> wrote:
>>>
>>> I like A. It will be simpler to maintain and evolve when we add the admin
>>> APIs.
>>>
>>>
>>>
>>>> On Fri, Jan 31, 2014 at 7:32 PM, Steve Morin <[EMAIL PROTECTED]>
>> wrote:
>>>>
>>>> I like A or C
>>>>
>>>> I think it will be clearer for people to separate the two.
>>>>
>>>>
>>>>> On Fri, Jan 31, 2014 at 3:29 PM, Jay Kreps <[EMAIL PROTECTED]>
>> wrote:
>>>>>
>>>>> For the new producer code we currently added the new module
>>>>>  clients
>>>>> This builds the jar kafka-clients.jar. The core module should be
>> renamed
>>>> to
>>>>> kafka-server and producer kafka-server.jar.
>>>>>
>>>>> It is the intention that the server will end up depending on the
>> clients
>>>>> but not vice versa (or we could make a separate module for that if we
>>>>> like).
>>>>>
>>>>> Integration code that tests clients against the server will live with
>> the
>>>>> server.
>>>>>
>>>>> There is some shared code between the clients and server. This is the
>>>>> kafka.common package. Currently this is in the clients module, which
>> is a
>>>>> little odd. We could alternatively break it into its own module, which
>>>>> might be nice. However I'm not sure it really warrants its own jar
>> since
>>>>> there isn't much point to that code on its own and having the clients
>>>> need
>>>>> two jars is kind of annoying. But maybe this doesn't matter because
>>>>> everyone just uses Maven?
>>>>>
>>>>> So the options are:
>>>>>
>>>>> A) Leave it the way it is: kafka-clients.jar and kafka-server.jar.
>>>>> B) Separate out the common code and have kafka-common.jar,
>>>>> kafka-clients.jar and kafka-server.jar (clients depends on common, and
>>>>> server depends on clients and common).
>>>>> C) Make a jar for each client. Currently this would be just producer
>> and
>>>>> consumer, but in the future we could add a more well-defined an Admin
>>>>> client for some of the administrative functions. If there ended up
>> being
>>>>> code that is client-specific but shared by multiple clients this could
>> be
>>>>> problematic.
>>>>>
>>>>> Alternately I don't know if the mapping from sub-modules to jars needs
>> to
>>>>> be one-to-one so we could seperate the clients and common code to
>> enforce
>>>>> compile dependencies and then glump it all into one client jar. That
>>>> might
>>>>> anger the build system, though.
>>>>>
>>>>> I think I vote for (A) but don't really care much.
>>>>>
>>>>> -Jay
>>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB