Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # dev - ProducerPool - KAFKA-240


Copy link to this message
-
ProducerPool - KAFKA-240
Joe Stein 2012-01-31, 05:36
I am not back from my length vacation until tomorrow (really Wednesday but
going to get back to things tomorrow night) but anyways I just started to
look at where I left off with the Producer Request changes....

Looking at the ProducerPool I am not sure what I am thinking of doing with
the brokerID is what others were thinking should be done or could be done
differently.

At first glance it seems best to make the ProducerPool from this

 class ProducerPoolData[V](topic: String,
                            bidPid: Partition,
                            data: Seq[V]) {

into this

 class ProducerPoolData(brokerID: Int,producerRequest: ProducerRequest)

and refactoring accordingly?

not sure what is lost by doing the message serialization prior [V] moving
up in the layers to this but have to get back digging in the code how that
would look

then

class ProducerData[K, V](private val topic: String,
                         private val key: K,
                         private val data: java.util.List[V]) {

would change to

class ProducerData[K](
                         private val key: K,
                         private val producerRequest: ProducerRequest) {

etc...

at first blush it almost seems like ProducerPoolData can go away and some
additions to producerRequest to handle things... but I am tired and need
some rest from my vacation :)

let me know, will jump back into this tomorrow night and then back again on
Thursday/Friday hopefully (including tests) can get the ProducerRequest
wrapped up by this coming Monday (if not sooner we'll see what other if any
hurdles come about)

Thanks!

/*
Joe Stein
http://www.linkedin.com/in/charmalloc
Twitter: @allthingshadoop <http://www.twitter.com/allthingshadoop>
*/