Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> Re: weird partitioner behavior


Copy link to this message
-
Re: weird partitioner behavior
I am still confused:

(1) What I observed is that not only the first msg got wrong partition
number, but also
     the second message.

(2) If kafka uses the first msg to get the correct msg. Does it mean that
if we use the
customized partitioner, we cannot get the right partition for this message?

Thanks,
Jane
On Fri, Jan 11, 2013 at 9:06 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> The very first message triggers the creation of the topic. Once the topic
> is created, the producer can know the # of partitions.
>
> Thanks,
>
> Jun
>
> On Fri, Jan 11, 2013 at 5:20 PM, Jane Wang <[EMAIL PROTECTED]> wrote:
>
> > I am using kafka 0.7. and I write a simple customized partitioner:
> >
> > public static class ExpertSearchCollectionPartitioner
> > implementsPartitioner<Long> {
> >
> >     @Override
> >
> >     public synchronized int partition(Long key, int numPartitions) {
> >
> >       System.out.print("    partition: " + key.hashCode() + ", " +
> > numPartitions + ", " + (key.hashCode() % numPartitions) + "\n");
> >
> >       return (int) (key.hashCode() % (long)numPartitions);
> >
> >     }
> >
> >   }
> >
> >
> > I got a weird problem. When I ran the producer, for the first one or two
> > message, the numPartitions is always 1. however, after that,
> >
> > it starts to get the value (96, which I config in server.property)
> >
> >
> > anyone has any clue? hint? suggestions?
> >
> >
> > Thanks!
> >
>