-Re: Leader not local for partition error?
Jun Rao 2013-07-31, 04:01
This is because we populate the key in
ReplicaManager.highWatermarkCheckpoints using the "dirs" config, but look
up the key using log.dir.getParent. So, if you have a trailing slash in the
config, they won't match. This seems a bug that we should fix. Could you
file a jira?
On Tue, Jul 30, 2013 at 9:18 AM, Maxime Petazzoni <[EMAIL PROTECTED]
> Nope, that's on a pretty standard GNU/Linux Debian system (jessie/sid)
> running a 3.9.8-1 kernel. But you were onto something. Removing the
> trailing slash is my log.dir config value made it work.
> I'm not sure why this would have an impact since the log directories seem
> to be correctly parsed as File objects in LogManager.scala:
> val logDirs: Array[File] = config.logDirs.map(new File(_)).toArray
> And in both cases the server logs reports that the log for 'test-0' was
> correctly loaded, which means that log directory is also correctly inserted
> into the logs Pool[TopicAndPartition, Log].
> That's about as far as my Scala knowledge goes though ;-) Let me know if
> you're able to reproduce the problem when you have a trailing slash as well.
> Maxime Petazzoni
> Sr. Platform Engineer
> m 408.310.0595
> From: Jun Rao [[EMAIL PROTECTED]]
> Sent: Monday, July 29, 2013 9:32 PM
> To: [EMAIL PROTECTED]
> Subject: Re: Leader not local for partition error?
> Are you on Windows? We have seen issues like that before on Windows. You
> may have to use "/" when configuring "log.dirs".
> On Mon, Jul 29, 2013 at 4:50 PM, Maxime Petazzoni <
> [EMAIL PROTECTED]
> > wrote:
> > Same issue with the 0.8 beta1 tarball. There is something interesting in
> > state-change.log though:
> > [2013-07-29 16:47:26,708] TRACE Broker 0 received LeaderAndIsr request
> > correlationId 6 from controller 0 epoch 1 starting the become-leader
> > transition for partition [test,0] (state.change.logger)
> > [2013-07-29 16:47:26,736] ERROR Error on broker 0 while processing
> > LeaderAndIsr request correlationId 6 received from controller 0 epoch 1
> > partition (test,0) (state.change.logger)
> > java.util.NoSuchElementException: key not found:
> > /home/maxime/opt/kafka/data/kafka
> > at scala.collection.MapLike$class.default(MapLike.scala:223)
> > at scala.collection.immutable.Map$Map1.default(Map.scala:93)
> > at scala.collection.MapLike$class.apply(MapLike.scala:134)
> > at scala.collection.immutable.Map$Map1.apply(Map.scala:93)
> > at kafka.cluster.Partition.getOrCreateReplica(Partition.scala:83)
> > ...
> > I have log.dir=/home/maxime/opt/kafka/data/kafka/ in server.properties.
> > That directory obviously exists after Kafka starts, and contains:
> > find /home/maxime/opt/kafka/data/kafka/
> > /home/maxime/opt/kafka/data/kafka/
> > /home/maxime/opt/kafka/data/kafka/test-0
> > /home/maxime/opt/kafka/data/kafka/test-0/00000000000000000000.log
> > /home/maxime/opt/kafka/data/kafka/test-0/00000000000000000000.index
> > /home/maxime/opt/kafka/data/kafka/.lock
> > Which is expected, given I have a single 'test' topic with a single
> > partition.
> > Any ideas? Can you reproduce the problem on your end with a freshly
> > extracted tarball?
> > Thanks,
> > /Max
> > --
> > Maxime Petazzoni
> > Sr. Platform Engineer
> > m 408.310.0595
> > www.turn.com
> > ________________________________________
> > From: Jun Rao [[EMAIL PROTECTED]]
> > Sent: Sunday, July 21, 2013 9:38 PM
> > To: [EMAIL PROTECTED]
> > Subject: Re: Leader not local for partition error?
> > Any error/exception in state-change or controller log? Also, could you
> > the 0.8 beta1 release?
> > Thanks,
> > Jun
> > On Mon, Jul 15, 2013 at 1:36 PM, Maxime Petazzoni <
> > [EMAIL PROTECTED]
> > > wrote:
> > > Hi all,
> > >
> > > I'm not sure if I'm doing something wrong or if I missed a step