Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Java 8 influence on next-generation Java producer and consumer APIs


Copy link to this message
-
Re: Java 8 influence on next-generation Java producer and consumer APIs
I think this is a good point and you are not the first person to bring it
up.

I am not hugely knowledgable about java 8 so any feedback would be useful.

In the producer I think the biggest impact is that the Callback can be
implemented with a lambda instead of a anon class which will be much nicer.
But maybe there are other things? Would be great if someone could take a
look at that producer api with a forward looking eye...

In the consumer there is potentially a lot of similarity between Stream and
the consumer. What we had tentatively proposed in this area was actually to
replace the iterator we currently provide with a lower-level method
   List<RecordAndMetadata> poll(long timeoutMs)
This method polls over all topics and returns a list of messages for
processing. This is arguably the least convenient end-user api for some of
the simpler cases, but the advantage is that it can be implemented in a
completely single threaded fashion which allows us to full decouple the
threading model of message processing--currently the zookeeper consumer
actually requires a per-topic blocking thread. There are a number of
slightly more convenient apis that could be implemented on top of this. I
think we were just about to kick off a more detailed discussion here...

-Jay
On Wed, Jan 29, 2014 at 12:05 PM, Clark Breyman <[EMAIL PROTECTED]> wrote:

> Jay et al,
>
> What are your current thoughts on ensuring that the next-generation APIs
> play nicely with both lambdas and the extensions to the standard runtime in
> Java 8?
>
> My thoughts are that if folks are doing the work to reimplement/redesign
> the API, it should be as compatible as possible with the latest stuff
> provided it doesn't break 1.6 compatibility (as long as that's required).
>
> Areas where this might influence the design:
> - Using single-method interfaces that could be supplied by lambdas when
> running in J8
> - Consideration of Spliterator, Supplier, Stream, ... when designing their
> Kakfa counterparts.
>
> Regards,
> Clark
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB