Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: partition numbers and mirrormaker


Copy link to this message
-
Re: partition numbers and mirrormaker
You can override the default number of partitions on the source and
target cluster to N. (num.partitions in the broker config). If you
need a higher (or lower) number of partitions for specific topics you
can override that on a per-topic basis (again, the config can be kept
identical between the source and target cluster).

Thanks,

Joel

On Tue, Jul 2, 2013 at 1:26 PM, Yu, Libo <[EMAIL PROTECTED]> wrote:
> Hi
>
> I asked this question previously and didn't get an answer.
> If a topic has N partitions on source cluster, after being mirrored, on the destination
> cluster it has only one partition.
>
> To avoid this issue, before mirroring, I create the same topic with N partitions on the
> Destination cluster. But then the mirrormaker cannot fetch all partitions and stop
> randomly.
>
> So what should we expect from mirrormake and what is the right way to use it? Thanks.
>
>
> Regards,
>
> Libo
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB