Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Kafka 0.8 - KeyedMessage?


+
Jason Huang 2013-01-07, 16:49
+
Jun Rao 2013-01-07, 17:04
Copy link to this message
-
Re: Kafka 0.8 - KeyedMessage?
Jun,

Thanks for the response. If I understand you correctly, messages with
the same key will not be automatically stored at the same partition
unless I implement a partition function to route the message based on
the key?

The quick start guide for 0.7 has the following:
"Send a message with a partition key. Messages with the same key are
sent to the same partition
ProducerData<String, String> data = new ProducerData<String,
String>("test-topic", "test-key", "test-message");
producer.send(data);"

So... there is no simple way to send messages to a certain default
partition in 0.8?

thanks,

Jason
On Mon, Jan 7, 2013 at 12:04 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> Jason,
>
> In 0.8, each message can optionally have a key. The key is retained as part
> of the message and will be stored in the broker. One can design a partition
> function to route the message based on the key. The default partitioner
> ignores the key and selects a partition at random.
>
> Thanks,
>
> Jun
>
> On Mon, Jan 7, 2013 at 8:49 AM, Jason Huang <[EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>> I did some search on the web but couldn't find any documentation for
>> 0.8 so I am trying to ask here:
>>
>> KeyedMessage is introduced in 0.8.0:
>> class KeyedMessage[K, V](val topic: String, val key: K, val message: V)
>>
>> Does the parameter "key" = "partition key"?
>>
>> If I build a KeyedMessage with a specific key string, then the message
>> will be stored to this partition, right?
>>
>> If I build a KeyedMessage without a key, then the message will be
>> distributed to a random partition belonging to this topic? Or will it
>> always use a certain default partition?
>>
>> thanks,
>>
>> Jason
>>

 
+
Neha Narkhede 2013-01-07, 18:52
+
Jason Huang 2013-01-07, 19:09
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB