Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Updated kafka client (producer and consumer)

Copy link to this message
Re: Updated kafka client (producer and consumer)
From the roadmap they published it looks like pipelining as part of the
client rewrite is happening post 0.8.

On Thu, Dec 5, 2013 at 3:52 PM, Tom Brown <[EMAIL PROTECTED]> wrote:

> In our environment we use currently use Kafka 0.7.1.
> The core features I am looking for in a client are this:
> 1. Provide confirmation of produce requests (or notification of
> disconnection during requests).
> 2. Uses asynchronous IO so that:
>   A. Multiple ops can be queued/in-flight at once.
>   B. Fetch #2 can be in-flight while fetch #1 is being processed (without
> requiring an extra thread per consumer)
> 3. Allow precise control of offsets (like the SimpleConsumer).
> I have been working on a version of the Kafka client that meets those
> requirements. However, I recall hearing talk of rewriting the client. Did
> this happen as part of 0.8.x? If so, how different is it from the 0.7.x
> clients, and how well would it support the requirements listed above?
> Thanks in advance!
> --Tom