Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> code layout


+
Jay Kreps 2014-01-31, 23:29
+
Steve Morin 2014-02-01, 03:33
+
Neha Narkhede 2014-02-01, 23:07
+
Guozhang Wang 2014-02-03, 00:14
+
Steve Morin 2014-02-03, 00:21
+
Jay Kreps 2014-02-03, 05:59
Copy link to this message
-
Re: code layout
That's why I was thinking c) makes sense from a design perspective.

> On Feb 2, 2014, at 21:59, Jay Kreps <[EMAIL PROTECTED]> wrote:
>
> Yes, this is a good point, the admin api could depend on zookeeper. It does
> today. That would not work in the existing clients package.
>
> -Jay
>
>
>> On Sun, Feb 2, 2014 at 4:20 PM, Steve Morin <[EMAIL PROTECTED]> wrote:
>>
>> Would the admin api's potentially have different dependencies?
>>
>>> On Feb 1, 2014, at 15:06, Neha Narkhede <[EMAIL PROTECTED]> wrote:
>>>
>>> I like A. It will be simpler to maintain and evolve when we add the admin
>>> APIs.
>>>
>>>
>>>
>>>> On Fri, Jan 31, 2014 at 7:32 PM, Steve Morin <[EMAIL PROTECTED]>
>> wrote:
>>>>
>>>> I like A or C
>>>>
>>>> I think it will be clearer for people to separate the two.
>>>>
>>>>
>>>>> On Fri, Jan 31, 2014 at 3:29 PM, Jay Kreps <[EMAIL PROTECTED]>
>> wrote:
>>>>>
>>>>> For the new producer code we currently added the new module
>>>>>  clients
>>>>> This builds the jar kafka-clients.jar. The core module should be
>> renamed
>>>> to
>>>>> kafka-server and producer kafka-server.jar.
>>>>>
>>>>> It is the intention that the server will end up depending on the
>> clients
>>>>> but not vice versa (or we could make a separate module for that if we
>>>>> like).
>>>>>
>>>>> Integration code that tests clients against the server will live with
>> the
>>>>> server.
>>>>>
>>>>> There is some shared code between the clients and server. This is the
>>>>> kafka.common package. Currently this is in the clients module, which
>> is a
>>>>> little odd. We could alternatively break it into its own module, which
>>>>> might be nice. However I'm not sure it really warrants its own jar
>> since
>>>>> there isn't much point to that code on its own and having the clients
>>>> need
>>>>> two jars is kind of annoying. But maybe this doesn't matter because
>>>>> everyone just uses Maven?
>>>>>
>>>>> So the options are:
>>>>>
>>>>> A) Leave it the way it is: kafka-clients.jar and kafka-server.jar.
>>>>> B) Separate out the common code and have kafka-common.jar,
>>>>> kafka-clients.jar and kafka-server.jar (clients depends on common, and
>>>>> server depends on clients and common).
>>>>> C) Make a jar for each client. Currently this would be just producer
>> and
>>>>> consumer, but in the future we could add a more well-defined an Admin
>>>>> client for some of the administrative functions. If there ended up
>> being
>>>>> code that is client-specific but shared by multiple clients this could
>> be
>>>>> problematic.
>>>>>
>>>>> Alternately I don't know if the mapping from sub-modules to jars needs
>> to
>>>>> be one-to-one so we could seperate the clients and common code to
>> enforce
>>>>> compile dependencies and then glump it all into one client jar. That
>>>> might
>>>>> anger the build system, though.
>>>>>
>>>>> I think I vote for (A) but don't really care much.
>>>>>
>>>>> -Jay
>>

 
+
Jun Rao 2014-02-04, 18:06