Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> Review Request 17263: New producer for Kafka.


Copy link to this message
-
Re: Review Request 17263: New producer for Kafka.
Yeah I will definitely do this. One todo is to work out the final config
names and document them all.

-Jay
On Wed, Jan 29, 2014 at 11:29 AM, Neha Narkhede <[EMAIL PROTECTED]>wrote:

>
>
> > On Jan. 26, 2014, 11:30 p.m., Sriram Subramanian wrote:
> > >
> clients/src/main/java/kafka/clients/producer/internals/RecordAccumulator.java,
> line 87
> > > <
> https://reviews.apache.org/r/17263/diff/1/?file=436455#file436455line87>
> > >
> > >     We can allocate a batch larger than the batchSize?
> >
> > Jay Kreps wrote:
> >     Yeah. This isn't terribly self-explanatory. The idea here is that
> you want to encourage medium sized batches to avoid small sends. So maybe
> you set a batch size of 4k. If you get a 1k message, you allocate a 4k
> buffer and append it and hope more messages show up. If you get a large
> message you just allocate a buffer for it and send it right away. This
> let's us have no artificial bound on message size, but handle small
> messages efficiently.
> >
> > Sriram Subramanian wrote:
> >     Makes sense
>
> Can we put the above explanation as a comment in the code? Will be easier
> to understand down the road.
>
>
> - Neha
>
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17263/#review32800
> -----------------------------------------------------------
>
>
> On Jan. 23, 2014, 8:54 p.m., Jay Kreps wrote:
> >
> > -----------------------------------------------------------
> > This is an automatically generated e-mail. To reply, visit:
> > https://reviews.apache.org/r/17263/
> > -----------------------------------------------------------
> >
> > (Updated Jan. 23, 2014, 8:54 p.m.)
> >
> >
> > Review request for kafka.
> >
> >
> > Bugs: KAFKA-1227
> >     https://issues.apache.org/jira/browse/KAFKA-1227
> >
> >
> > Repository: kafka
> >
> >
> > Description
> > -------
> >
> > KAFKA-1227 New producer!
> >
> >
> > Diffs
> > -----
> >
> >   clients/build.sbt PRE-CREATION
> >
> clients/src/main/java/kafka/clients/producer/BufferExhaustedException.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/Callback.java PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/DefaultPartitioner.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/KafkaProducer.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/MockProducer.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/Partitioner.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/Producer.java PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/ProducerConfig.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/ProducerRecord.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/RecordSend.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/internals/BufferPool.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/internals/Metadata.java
> PRE-CREATION
> >
> clients/src/main/java/kafka/clients/producer/internals/ProduceRequestResult.java
> PRE-CREATION
> >
> clients/src/main/java/kafka/clients/producer/internals/RecordAccumulator.java
> PRE-CREATION
> >
> clients/src/main/java/kafka/clients/producer/internals/RecordBatch.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/producer/internals/Sender.java
> PRE-CREATION
> >   clients/src/main/java/kafka/clients/tools/ProducerPerformance.java
> PRE-CREATION
> >   clients/src/main/java/kafka/common/ByteSerialization.java PRE-CREATION
> >   clients/src/main/java/kafka/common/Cluster.java PRE-CREATION
> >   clients/src/main/java/kafka/common/Configurable.java PRE-CREATION
> >   clients/src/main/java/kafka/common/Deserializer.java PRE-CREATION
> >   clients/src/main/java/kafka/common/KafkaException.java PRE-CREATION
> >   clients/src/main/java/kafka/common/Metric.java PRE-CREATION
> >   clients/src/main/java/kafka/common/Node.java PRE-CREATION