Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # dev >> understanding partitions based on wiki example of profile visits

S Ahmed 2012-11-26, 03:54
Copy link to this message
Re: understanding partitions based on wiki example of profile visits
sorry wrong list.
On Sun, Nov 25, 2012 at 10:54 PM, S Ahmed <[EMAIL PROTECTED]> wrote:

> The wiki states "Consider an application that would like to maintain an
> aggregation of the number of profile visitors for each member. It would
> like to send all profile visit events for a member to a particular
> partition and, hence, have all updates for a member to appear in the same
> stream for the same consumer thread." (
> http://incubator.apache.org/kafka/design.html)
> So say I have 5 broker servers, now my producer will send a message for a
> particular profile page visit, with the default algorithm using hash(member_id)%num_partitions
> to figur out which broker server to send it it.
> So a particular members pageview messages will all go to a single server
> then, is this the case?  And therefore all the messages for a given user
> will be in the correct order also right?
> So a consumer group that subscribes to the 'profile-page-view' topic will
> consume page view related messages, is it possible to subscribe to a
> particular broker partition also?
> Are broker partitions meant for cases when you want all messages to be
> saved on the same node?
Jay Kreps 2012-11-26, 16:53
S Ahmed 2012-11-26, 17:19
Jay Kreps 2012-11-26, 19:02
S Ahmed 2012-11-27, 20:00
Jay Kreps 2012-11-27, 21:35