Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Kafka 0.8 delete log failed


Copy link to this message
-
Re: Kafka 0.8 delete log failed
Jay Kreps 2013-12-11, 17:54
Is the path d:\kafka-logs\test001-0\00000000000000507600.index correct?

The tricky thing here is we don't have access to windows for testing so we
will need a bit more help for debugging. If you write a simple Java program
that does
  System.out.println(new File("d:\kafka-logs\test001-0\
00000000000000507600.index").delete())
Does it work?

One irritating thing about the delete API in Java is that it doesn't give
an error so this can happen do to the file not existing or due to
permissions and perhaps on Windows due to locking.
On Wed, Dec 11, 2013 at 12:30 AM, CuiLiang <[EMAIL PROTECTED]> wrote:

> Update this issue.
> I update the log config to log.dirs=\\kafka-logs. The log file is deleted
> but still can't delete the index file. I got below error message.
>
> [2013-12-11 00:07:59,671] INFO Deleting index
> d:\kafka-logs\test001-0\00000000000000507600.index (kafka.log.OffsetIndex)
> [2013-12-11 00:07:59,672] ERROR error in loggedRunnable
> (kafka.utils.Utils$)
> kafka.common.KafkaStorageException: Deleting log segment 507600 failed.
> at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:613)
> at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:608)
> at
>
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
> at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32)
> at kafka.log.Log.deleteSegments(Log.scala:608)
> at
>
> kafka.log.LogManager.kafka$log$LogManager$$cleanupSegmentsToMaintainSize(LogManager.scala:264)
> at kafka.log.LogManager$$anonfun$cleanupLogs$2.apply(LogManager.scala:277)
> at kafka.log.LogManager$$anonfun$cleanupLogs$2.apply(LogManager.scala:275)
> at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> at
>
> scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
> at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
> at
>
> scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:521)
> at kafka.log.LogManager.cleanupLogs(LogManager.scala:275)
> at
> kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:141)
> at kafka.utils.Utils$$anon$2.run(Utils.scala:68)
> at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
> at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> Source)
> at
>
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> Source)
> at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> at java.lang.Thread.run(Unknown Source)
>
> Thanks,
> Liang Cui
>
>
> 2013/12/11 CuiLiang <[EMAIL PROTECTED]>
>
> > The Kafka folder is d:\kafka_2.8.0-0.8.0. Kafka log folder is
> > d:\data\kafka-logs. kafka batch folder is d:\kafka_2.8.0-0.8.0\bin.
> > log.dirs=..\\..\\data\\kafka-logs
> > The log segments files is created correct, but can't delete the log file.
> > My machine has C, D, E,F, G, k partitions.
> >
> >
> > Thanks,
> > Liang Cui
> >
> >
> > 2013/12/11 Jay Kreps <[EMAIL PROTECTED]>
> >
> >> What is your configuration for data.dirs (the path where data is) and
> what
> >> is the set of disks/volumes on the machine?
> >>
> >> -Jay
> >>
> >>
> >> On Tue, Dec 10, 2013 at 12:50 AM, CuiLiang <[EMAIL PROTECTED]>
> wrote:
> >>
> >> > Hi All,
> >> >
> >> > I'm use Kafka 0.8 release build with 1 partition, 1 replica. My OS is
> >> > Windows server 2012 and JDK is 1.7. I got below error when Kafka
> delete
> >> > logs. Any guidance would be of great help.
> >> >
> >> > [2013-12-09 04:00:10,525] ERROR error in loggedRunnable
> >> > (kafka.utils.Utils$)kafka.common.KafkaStorageException: Deleting log
> >> > segment 140332200 failed. at
> >> > kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:613) at
> >> > kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:608) at