Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Updated kafka client (producer and consumer)


Copy link to this message
-
Re: Updated kafka client (producer and consumer)
Timothy Chen 2013-12-05, 23:54
From the roadmap they published it looks like pipelining as part of the
client rewrite is happening post 0.8.

Tim
On Thu, Dec 5, 2013 at 3:52 PM, Tom Brown <[EMAIL PROTECTED]> wrote:

> In our environment we use currently use Kafka 0.7.1.
>
> The core features I am looking for in a client are this:
>
> 1. Provide confirmation of produce requests (or notification of
> disconnection during requests).
> 2. Uses asynchronous IO so that:
>   A. Multiple ops can be queued/in-flight at once.
>   B. Fetch #2 can be in-flight while fetch #1 is being processed (without
> requiring an extra thread per consumer)
> 3. Allow precise control of offsets (like the SimpleConsumer).
>
> I have been working on a version of the Kafka client that meets those
> requirements. However, I recall hearing talk of rewriting the client. Did
> this happen as part of 0.8.x? If so, how different is it from the 0.7.x
> clients, and how well would it support the requirements listed above?
>
> Thanks in advance!
>
> --Tom
>