Kafka, mail # user - Re: Java 8 influence on next-generation Java producer and consumer APIs - 2014-01-30, 18:14
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Java 8 influence on next-generation Java producer and consumer APIs
I think this is a good point and you are not the first person to bring it
up.

I am not hugely knowledgable about java 8 so any feedback would be useful.

In the producer I think the biggest impact is that the Callback can be
implemented with a lambda instead of a anon class which will be much nicer.
But maybe there are other things? Would be great if someone could take a
look at that producer api with a forward looking eye...

In the consumer there is potentially a lot of similarity between Stream and
the consumer. What we had tentatively proposed in this area was actually to
replace the iterator we currently provide with a lower-level method
   List<RecordAndMetadata> poll(long timeoutMs)
This method polls over all topics and returns a list of messages for
processing. This is arguably the least convenient end-user api for some of
the simpler cases, but the advantage is that it can be implemented in a
completely single threaded fashion which allows us to full decouple the
threading model of message processing--currently the zookeeper consumer
actually requires a per-topic blocking thread. There are a number of
slightly more convenient apis that could be implemented on top of this. I
think we were just about to kick off a more detailed discussion here...

-Jay
On Wed, Jan 29, 2014 at 12:05 PM, Clark Breyman <[EMAIL PROTECTED]> wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB