Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka, mail # user - Kafka 0.8 - KeyedMessage?


+
Jason Huang 2013-01-07, 16:49
+
Jun Rao 2013-01-07, 17:04
Copy link to this message
-
Re: Kafka 0.8 - KeyedMessage?
Jason Huang 2013-01-07, 17:31
Jun,

Thanks for the response. If I understand you correctly, messages with
the same key will not be automatically stored at the same partition
unless I implement a partition function to route the message based on
the key?

The quick start guide for 0.7 has the following:
"Send a message with a partition key. Messages with the same key are
sent to the same partition
ProducerData<String, String> data = new ProducerData<String,
String>("test-topic", "test-key", "test-message");
producer.send(data);"

So... there is no simple way to send messages to a certain default
partition in 0.8?

thanks,

Jason
On Mon, Jan 7, 2013 at 12:04 PM, Jun Rao <[EMAIL PROTECTED]> wrote:
> Jason,
>
> In 0.8, each message can optionally have a key. The key is retained as part
> of the message and will be stored in the broker. One can design a partition
> function to route the message based on the key. The default partitioner
> ignores the key and selects a partition at random.
>
> Thanks,
>
> Jun
>
> On Mon, Jan 7, 2013 at 8:49 AM, Jason Huang <[EMAIL PROTECTED]> wrote:
>
>> Hello,
>>
>> I did some search on the web but couldn't find any documentation for
>> 0.8 so I am trying to ask here:
>>
>> KeyedMessage is introduced in 0.8.0:
>> class KeyedMessage[K, V](val topic: String, val key: K, val message: V)
>>
>> Does the parameter "key" = "partition key"?
>>
>> If I build a KeyedMessage with a specific key string, then the message
>> will be stored to this partition, right?
>>
>> If I build a KeyedMessage without a key, then the message will be
>> distributed to a random partition belonging to this topic? Or will it
>> always use a certain default partition?
>>
>> thanks,
>>
>> Jason
>>

 
+
Neha Narkhede 2013-01-07, 18:52
+
Jason Huang 2013-01-07, 19:09