Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> New Producer Public API


Copy link to this message
-
Re: New Producer Public API
Hey Chris,

1. My thinking was that it represents the in progress sending of a record,
but I take your point. It sounds like the consensus is that it would be
better to have the return type be something like Future<RecordMetadata>
where the metadata has the offset/topic/partition and the Future has the
exception if there is one.

2. Yeah I totally agree. The question here is whether we provide constants
for the config strings or just refer people to the docs. People have
requested constants. There are some issues. It means that removing configs
is not backwards compatible. That javadoc does give the string of constant
values.

3-6. Agree, all of these need to be reconsidered more carefully.

7. Fixed.

8. Ack, actually the problem is that that should be under internals and not
in the generated javadocs at all. That is an internal class, what you
actually get is a RecordSend.

9. Yeah it's an internal class so it maps more closely to the response from
the server. The server doesn't give the offset for each record but rather
just the base offset since the offsets are guaranteed to be sequential.

10. This is a point worth discussing. I have used the scala style method()
rather than the java style getMethod()/setMethod(). My objection to getters
and setters is that they imply something about the implementation (namely
that you are getting and setting something) and somehow because of that
naming people tend to make one getter/setter per member variable instead of
exposing the higher level operations on the type. I realize this is at odds
with basically the rest of the Java code in the world. Either way let's
pick a style and use it throughout.

11. Cool.

12. Cool.

13. Cool.

-Jay

On Wed, Jan 29, 2014 at 2:41 PM, Chris Riccomini <[EMAIL PROTECTED]>wrote:

> Hey Guys,
>
> My 2c.
>
> 1. RecordSend is a confusing name to me. Shouldn't it be
> RecordSendResponse?
> 2. Random nit: it's annoying to have the Javadoc info for the contstants
> on
> http://empathybox.com/kafka-javadoc/kafka/clients/producer/ProducerConfig.h
> tml<http://empathybox.com/kafka-javadoc/kafka/clients/producer/ProducerConfig.html>,
> but the string constant values on
> http://empathybox.com/kafka-javadoc/constant-values.html#kafka.clients.prod
> ucer.ProducerConfig.MAX_REQUEST_SIZE_CONFIG. Find myself toggling between
> the two a lot. Not sure if this can be fixed easily.
> 3. MAX_PARTITION_SIZE_CONFIG - this name is kind of confusing.
> Specifically, use of the term "partition". thought it was related to Kafka
> topic partitions, not grouping together/batching.
> 4. METADATA_FETCH_TIMEOUT_CONFIG - what happens if this timeout is
> exceeded? Do we get an exception on send()?
> 5. METADATA_REFRESH_MS_CONFIG - why is this useful? Has to do with acks=0,
> right? Worth documenting, I think.
> 6. PARTITIONER_CLASS_CONFIG - link to partitioner interface in javadocs.
> Also, missing a period.
> 7. KafkaProducer.html - send() documentation says "archive" when you mean
> achieve, I think.
> 8. No javadoc for ProduceRequestResult.
> 9. In ProduceRequestResult, I understand baseOffset to be the first offset
> of the set. Is it possible to get the last offset, as well? If I send
> messages A, B, C, D, I'm most interested in D's offset.
> 10. In ProduceRequestResult, prefer Java-bean style (getError,
> isCompleted).
> 11. At first glance, I like option 1A in your serialization list.
> 12. We should definitely not introduce a ZK dependency for bootstrapping
> broker host/ports.
> 13. No favor on the Future discussion. I really^Int.Max hate checked
> exceptions, but I also like standard interfaces. It's a wash in my book.
>
>
> Cheers,
> Chris
>
> On 1/29/14 10:34 AM, "Neha Narkhede" <[EMAIL PROTECTED]> wrote:
>
> >>> The challenge of directly exposing ProduceRequestResult is that the
> >offset
> >provided is just the base offset and there is no way to know for a
> >particular message where it was in relation to that base offset because
> >the
> >batching is transparent and non-deterministic.

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB