Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> C/C++ Client


Copy link to this message
-
Re: C/C++ Client
I'm on OSX. It's not going to production on OSX, but I have a requirement
that the entire stack be deployable on a developer's machine. OSX does have
TLS, but not the local variable __thread. It would need to be reworked to
use the pthreads API. That by itself isn't much work it's just I don't know
what else is going to need to be fixed.
On Thu, Mar 28, 2013 at 7:38 AM, Magnus Edenhill <[EMAIL PROTECTED]> wrote:

> Hi Matt,
>
> regarding librdkafka:
>
> - the AF_NETLINK stuff has now been removed, a helper function had support
> for it but it had no relevance to Kafka operation.
>
> - TLS: it would be possible to remove the need for TLS in rdkafka, but it
> does feel like a leap back in time; even NetBSD has support for TLS now
> (since 6.0)! What system are you on?
>
> - 0.8: rdkafka will have 0.8 protocol support about the same time as Apache
> Kafka 0.8 is released, which is within a month from what I understand.
>
> - C++: I'd be happy to accept patches/pull-reqs to make rdkafka more C++
> friendly.
>
> Regards,
> Magnus
>
>
>
> 2013/3/28 Matthew Stump <[EMAIL PROTECTED]>
>
> > Howdy,
> >
> > I'm considering the use of Kafka in the rewrite of a big legacy product.
> A
> > good chunk of the back end code is going to be written in C++ (large in
> > memory data-structures). The two possible options available to me for
> > clients appear to be:
> >
> > https://github.com/edenhill/librdkafka
> >
> > and
> >
> > https://github.com/quipo/kafka-cpp
> >
> > The problem is that librdkafka currently only works on Linux due to the
> use
> > of the AF_NETLINK API, and thread local storage. There may be other
> issues,
> > but I just started playing with it today and that's what I've discovered
> > thus far.
> >
> > kafka-cpp is incomplete (no consumer) and it looks unused.
> >
> > For either I would need to hop in and do some significant work. Is there
> > any client I'm missing that can shorten my path?
> >
> > If I adopt one of these projects (lets say kafka-cpp) am I better off
> > implementing the 0.8 protocol? I'de like to have something running in
> > staging a couple months from now. How far out is 0.8?
> >
> > Thanks,
> > Matt Stump
> >
>