Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> Review Request 17263: New producer for Kafka.


Copy link to this message
-
Re: Review Request 17263: New producer for Kafka.

-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/17263/#review32806
-----------------------------------------------------------

clients/src/main/java/kafka/clients/producer/internals/BufferPool.java
<https://reviews.apache.org/r/17263/#comment61815>

    Ok makes sense. I am wondering if max memory needs to be hard bound or soft limit and just allocate the buffer for requests that are larger than the pool size and reduce complexity in this code. Not recommending anything specific.

clients/src/main/java/kafka/clients/producer/internals/BufferPool.java
<https://reviews.apache.org/r/17263/#comment61814>

    I think there is a case here where two threads could just be accumulating memory and not reach their require capacity and always be in the loop. I dont think Condition.signal guarantees any order in which threads can be released.
- Sriram Subramanian
On Jan. 23, 2014, 8:54 p.m., Jay Kreps wrote:
>
> -----------------------------------------------------------
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/17263/
> -----------------------------------------------------------
>
> (Updated Jan. 23, 2014, 8:54 p.m.)
>
>
> Review request for kafka.
>
>
> Bugs: KAFKA-1227
>     https://issues.apache.org/jira/browse/KAFKA-1227
>
>
> Repository: kafka
>
>
> Description
> -------
>
> KAFKA-1227 New producer!
>
>
> Diffs
> -----
>
>   clients/build.sbt PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/BufferExhaustedException.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/Callback.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/DefaultPartitioner.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/KafkaProducer.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/MockProducer.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/Partitioner.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/Producer.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/ProducerConfig.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/ProducerRecord.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/RecordSend.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/internals/BufferPool.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/internals/Metadata.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/internals/ProduceRequestResult.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/internals/RecordAccumulator.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/internals/RecordBatch.java PRE-CREATION
>   clients/src/main/java/kafka/clients/producer/internals/Sender.java PRE-CREATION
>   clients/src/main/java/kafka/clients/tools/ProducerPerformance.java PRE-CREATION
>   clients/src/main/java/kafka/common/ByteSerialization.java PRE-CREATION
>   clients/src/main/java/kafka/common/Cluster.java PRE-CREATION
>   clients/src/main/java/kafka/common/Configurable.java PRE-CREATION
>   clients/src/main/java/kafka/common/Deserializer.java PRE-CREATION
>   clients/src/main/java/kafka/common/KafkaException.java PRE-CREATION
>   clients/src/main/java/kafka/common/Metric.java PRE-CREATION
>   clients/src/main/java/kafka/common/Node.java PRE-CREATION
>   clients/src/main/java/kafka/common/PartitionInfo.java PRE-CREATION
>   clients/src/main/java/kafka/common/Serializer.java PRE-CREATION
>   clients/src/main/java/kafka/common/StringSerialization.java PRE-CREATION
>   clients/src/main/java/kafka/common/TopicPartition.java PRE-CREATION
>   clients/src/main/java/kafka/common/config/AbstractConfig.java PRE-CREATION
>   clients/src/main/java/kafka/common/config/ConfigDef.java PRE-CREATION
>   clients/src/main/java/kafka/common/config/ConfigException.java PRE-CREATION