-Re: Loading kafka server within a Java process
Eric Tschetter 2013-02-14, 23:08
Consumer + broker embedding can be done, I believe. It's just
producer+broker embedding that they don't like :).
On Thu, Feb 14, 2013 at 4:51 PM, Jamie Wang <[EMAIL PROTECTED]> wrote:
> Thank you both for the info. I'll keep my eye open since I do have a need
> to embed a consumer in the same process.
> -----Original Message-----
> From: Eric Tschetter [mailto:[EMAIL PROTECTED]]
> Sent: Thursday, February 14, 2013 2:44 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Loading kafka server within a Java process
> As Jay says, this is definitely possible. We also do it at Metamarkets.
> The one thing that we found odd when doing it though is that you cannot (or
> couldn't with the version we are using, I'm not sure about 0.8) embed a
> producer that produces for an embedded broker. You instead have to have
> your producer talk over the "network" to the broker even though it is
> running inside the same process. We haven't run into any issues doing this
> yet, but it's something to be aware of.
> On Thu, Feb 14, 2013 at 4:39 PM, Jay Kreps <[EMAIL PROTECTED]> wrote:
> > Yes, it will work. We do this at LinkedIn, actually.
> > -Jay
> > On Thu, Feb 14, 2013 at 12:15 PM, Jamie Wang <[EMAIL PROTECTED]>
> > wrote:
> > > I have a situation where we have constraint on the number of processes
> > we can use and hence, is it possible to load kafka server within another
> > Java process? Looking at the kafka.kafka it seems that I can create a
> > object that does most of what Kafka.kafka.main() do. Any thoughts if
> > will work? What are some of the things I need to watch out? And is there
> > another better way to do this? Thank you for your time.
> > >
> > > Jamie