Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Java 8 influence on next-generation Java producer and consumer APIs

Copy link to this message
Re: Java 8 influence on next-generation Java producer and consumer APIs
I think this is a good point and you are not the first person to bring it

I am not hugely knowledgable about java 8 so any feedback would be useful.

In the producer I think the biggest impact is that the Callback can be
implemented with a lambda instead of a anon class which will be much nicer.
But maybe there are other things? Would be great if someone could take a
look at that producer api with a forward looking eye...

In the consumer there is potentially a lot of similarity between Stream and
the consumer. What we had tentatively proposed in this area was actually to
replace the iterator we currently provide with a lower-level method
   List<RecordAndMetadata> poll(long timeoutMs)
This method polls over all topics and returns a list of messages for
processing. This is arguably the least convenient end-user api for some of
the simpler cases, but the advantage is that it can be implemented in a
completely single threaded fashion which allows us to full decouple the
threading model of message processing--currently the zookeeper consumer
actually requires a per-topic blocking thread. There are a number of
slightly more convenient apis that could be implemented on top of this. I
think we were just about to kick off a more detailed discussion here...

On Wed, Jan 29, 2014 at 12:05 PM, Clark Breyman <[EMAIL PROTECTED]> wrote:

> Jay et al,
> What are your current thoughts on ensuring that the next-generation APIs
> play nicely with both lambdas and the extensions to the standard runtime in
> Java 8?
> My thoughts are that if folks are doing the work to reimplement/redesign
> the API, it should be as compatible as possible with the latest stuff
> provided it doesn't break 1.6 compatibility (as long as that's required).
> Areas where this might influence the design:
> - Using single-method interfaces that could be supplied by lambdas when
> running in J8
> - Consideration of Spliterator, Supplier, Stream, ... when designing their
> Kakfa counterparts.
> Regards,
> Clark