Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # dev - code layout


Copy link to this message
-
Re: code layout
Jay Kreps 2014-02-03, 05:59
Yes, this is a good point, the admin api could depend on zookeeper. It does
today. That would not work in the existing clients package.

-Jay
On Sun, Feb 2, 2014 at 4:20 PM, Steve Morin <[EMAIL PROTECTED]> wrote:

> Would the admin api's potentially have different dependencies?
>
> > On Feb 1, 2014, at 15:06, Neha Narkhede <[EMAIL PROTECTED]> wrote:
> >
> > I like A. It will be simpler to maintain and evolve when we add the admin
> > APIs.
> >
> >
> >
> >> On Fri, Jan 31, 2014 at 7:32 PM, Steve Morin <[EMAIL PROTECTED]>
> wrote:
> >>
> >> I like A or C
> >>
> >> I think it will be clearer for people to separate the two.
> >>
> >>
> >>> On Fri, Jan 31, 2014 at 3:29 PM, Jay Kreps <[EMAIL PROTECTED]>
> wrote:
> >>>
> >>> For the new producer code we currently added the new module
> >>>   clients
> >>> This builds the jar kafka-clients.jar. The core module should be
> renamed
> >> to
> >>> kafka-server and producer kafka-server.jar.
> >>>
> >>> It is the intention that the server will end up depending on the
> clients
> >>> but not vice versa (or we could make a separate module for that if we
> >>> like).
> >>>
> >>> Integration code that tests clients against the server will live with
> the
> >>> server.
> >>>
> >>> There is some shared code between the clients and server. This is the
> >>> kafka.common package. Currently this is in the clients module, which
> is a
> >>> little odd. We could alternatively break it into its own module, which
> >>> might be nice. However I'm not sure it really warrants its own jar
> since
> >>> there isn't much point to that code on its own and having the clients
> >> need
> >>> two jars is kind of annoying. But maybe this doesn't matter because
> >>> everyone just uses Maven?
> >>>
> >>> So the options are:
> >>>
> >>> A) Leave it the way it is: kafka-clients.jar and kafka-server.jar.
> >>> B) Separate out the common code and have kafka-common.jar,
> >>> kafka-clients.jar and kafka-server.jar (clients depends on common, and
> >>> server depends on clients and common).
> >>> C) Make a jar for each client. Currently this would be just producer
> and
> >>> consumer, but in the future we could add a more well-defined an Admin
> >>> client for some of the administrative functions. If there ended up
> being
> >>> code that is client-specific but shared by multiple clients this could
> be
> >>> problematic.
> >>>
> >>> Alternately I don't know if the mapping from sub-modules to jars needs
> to
> >>> be one-to-one so we could seperate the clients and common code to
> enforce
> >>> compile dependencies and then glump it all into one client jar. That
> >> might
> >>> anger the build system, though.
> >>>
> >>> I think I vote for (A) but don't really care much.
> >>>
> >>> -Jay
> >>
>