Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - C/C++ Client


+
Matthew Stump 2013-03-28, 03:49
+
Magnus Edenhill 2013-03-28, 14:51
+
Rohit Prasad 2013-03-28, 15:34
+
mrevilgnome 2013-03-28, 15:53
Copy link to this message
-
Re: C/C++ Client
mrevilgnome 2013-03-28, 15:52
I'm on OSX. It's not going to production on OSX, but I have a requirement
that the entire stack be deployable on a developer's machine. OSX does have
TLS, but not the local variable __thread. It would need to be reworked to
use the pthreads API. That by itself isn't much work it's just I don't know
what else is going to need to be fixed.
On Thu, Mar 28, 2013 at 7:38 AM, Magnus Edenhill <[EMAIL PROTECTED]> wrote:

> Hi Matt,
>
> regarding librdkafka:
>
> - the AF_NETLINK stuff has now been removed, a helper function had support
> for it but it had no relevance to Kafka operation.
>
> - TLS: it would be possible to remove the need for TLS in rdkafka, but it
> does feel like a leap back in time; even NetBSD has support for TLS now
> (since 6.0)! What system are you on?
>
> - 0.8: rdkafka will have 0.8 protocol support about the same time as Apache
> Kafka 0.8 is released, which is within a month from what I understand.
>
> - C++: I'd be happy to accept patches/pull-reqs to make rdkafka more C++
> friendly.
>
> Regards,
> Magnus
>
>
>
> 2013/3/28 Matthew Stump <[EMAIL PROTECTED]>
>
> > Howdy,
> >
> > I'm considering the use of Kafka in the rewrite of a big legacy product.
> A
> > good chunk of the back end code is going to be written in C++ (large in
> > memory data-structures). The two possible options available to me for
> > clients appear to be:
> >
> > https://github.com/edenhill/librdkafka
> >
> > and
> >
> > https://github.com/quipo/kafka-cpp
> >
> > The problem is that librdkafka currently only works on Linux due to the
> use
> > of the AF_NETLINK API, and thread local storage. There may be other
> issues,
> > but I just started playing with it today and that's what I've discovered
> > thus far.
> >
> > kafka-cpp is incomplete (no consumer) and it looks unused.
> >
> > For either I would need to hop in and do some significant work. Is there
> > any client I'm missing that can shorten my path?
> >
> > If I adopt one of these projects (lets say kafka-cpp) am I better off
> > implementing the 0.8 protocol? I'de like to have something running in
> > staging a couple months from now. How far out is 0.8?
> >
> > Thanks,
> > Matt Stump
> >
>

 
+
Jun Rao 2013-04-01, 14:54
+
mrevilgnome 2013-04-01, 15:23
+
Simon Whitear 2013-04-16, 18:45