Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Re: partition numbers and mirrormaker


Copy link to this message
-
Re: partition numbers and mirrormaker
Joel Koshy 2013-07-02, 21:23
You can override the default number of partitions on the source and
target cluster to N. (num.partitions in the broker config). If you
need a higher (or lower) number of partitions for specific topics you
can override that on a per-topic basis (again, the config can be kept
identical between the source and target cluster).

Thanks,

Joel

On Tue, Jul 2, 2013 at 1:26 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:
> Hi
>
> I asked this question previously and didn't get an answer.
> If a topic has N partitions on source cluster, after being mirrored, on the destination
> cluster it has only one partition.
>
> To avoid this issue, before mirroring, I create the same topic with N partitions on the
> Destination cluster. But then the mirrormaker cannot fetch all partitions and stop
> randomly.
>
> So what should we expect from mirrormake and what is the right way to use it? Thanks.
>
>
> Regards,
>
> Libo
>