Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Leader doesn't get assigned for new topics


Copy link to this message
-
Re: Leader doesn't get assigned for new topics
Neha Narkhede 2013-09-13, 22:27
Ah ok. Thanks for sharing that.

On Fri, Sep 13, 2013 at 2:50 PM, Rajasekar Elango <[EMAIL PROTECTED]>wrote:

> We have 3 zookeeper node in the cluster with a hardware load balancer .  In
> one of the zookeeper, we did not configure ensemble correctly (server.n
> property in zoo.cfg) . So it ended up as like 2 nodes in one cluster, one
> node in other cluster. The load balancer is randomly hitting one of 2
> zookeepers in two different cluster.
>
> Thanks,
> Raja.
>
>
> On Fri, Sep 13, 2013 at 1:04 PM, Neha Narkhede <[EMAIL PROTECTED]
> >wrote:
>
> > Just curious to know, what was the misconfiguration?
> >
> >
> > On Fri, Sep 13, 2013 at 10:02 AM, Rajasekar Elango
> > <[EMAIL PROTECTED]>wrote:
> >
> > > Thanks Neha and Jun, It turned out to be miss configuration in our
> > > zookeeper cluster. After correcting it everything looks good.
> > >
> > > Thanks,
> > > Raja.
> > >
> > >
> > > On Fri, Sep 13, 2013 at 10:13 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > Any error in the controller and the state-change log? Are brokers
> 2,3,4
> > > > alive?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Thu, Sep 12, 2013 at 4:56 PM, Rajasekar Elango <
> > > [EMAIL PROTECTED]
> > > > >wrote:
> > > >
> > > > > We are seeing a problem that we we try to send messages to new
> topic
> > it
> > > > > fails kafka.common.LeaderNotAvailableException. But usually this
> > > problem
> > > > > will be transient and if we re-send messages to same topic will
> work.
> > > But
> > > > > now we tried rending message to same topic several time, but still
> > > fails
> > > > > with same error:
> > > > >
> > > > > In the server log I see ] Auto creation of topic test-sjl2 with 8
> > > > > partitions and replication factor 3 is successful!. But listTopics
> > > > command
> > > > > shows leader "none" like below:
> > > > >
> > > > > topic: test-sjl2        partition: 0    leader: none    replicas:
> > 2,4,3
> > > > > isr:
> > > > > topic: test-sjl2        partition: 1    leader: none    replicas:
> > 3,2,4
> > > > > isr:
> > > > > topic: test-sjl2        partition: 2    leader: none    replicas:
> > 4,3,2
> > > > > isr:
> > > > > topic: test-sjl2        partition: 3    leader: none    replicas:
> > 2,3,4
> > > > > isr:
> > > > > topic: test-sjl2        partition: 4    leader: none    replicas:
> > 3,4,2
> > > > > isr:
> > > > > topic: test-sjl2        partition: 5    leader: none    replicas:
> > 4,2,3
> > > > > isr:
> > > > > topic: test-sjl2        partition: 6    leader: none    replicas:
> > 2,4,3
> > > > > isr:
> > > > > topic: test-sjl2        partition: 7    leader: none    replicas:
> > 3,2,4
> > > > > isr:
> > > > >
> > > > > I also see following NotLeaderForPatritionExcetion and
> > > ZookeeperExcetion
> > > > in
> > > > > logs
> > > > >
> > > > > kafka.common.NotLeaderForPartitionException
> > > > >         at
> > > sun.reflect.GeneratedConstructorAccessor19.newInstance(Unknown
> > > > > Source)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:27)
> > > > >         at
> > > > java.lang.reflect.Constructor.newInstance(Constructor.java:513)
> > > > >         at java.lang.Class.newInstance0(Class.java:355)
> > > > >         at java.lang.Class.newInstance(Class.java:308)
> > > > >         at
> > > kafka.common.ErrorMapping$.exceptionFor(ErrorMapping.scala:70)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$4$$anonfun$apply$5.apply(AbstractFetcherThread.scala:158)
> > > > >         at
> > > > >
> > > > >
> > > >
> > >
> >
> kafka.server.AbstractFetcherThread$$anonfun$processFetchRequest$4$$anonfun$apply$5.apply(AbstractFetcherThread.scala:158)
> > > > >         at kafka.utils.Logging$class.warn(Logging.scala:88)
> > > > >         at
> > > > kafka.utils.ShutdownableThread.warn(ShutdownableThread.scala:23)
> > > > >         at