Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka 0.8 delete log failed


Copy link to this message
-
Re: Kafka 0.8 delete log failed
I tried the sample code and it works. I also can delete the old index file
manually.

Thanks,
Liang Cui
2013/12/12 Jay Kreps <[EMAIL PROTECTED]>

> Is the path d:\kafka-logs\test001-0\00000000000000507600.index correct?
>
> The tricky thing here is we don't have access to windows for testing so we
> will need a bit more help for debugging. If you write a simple Java program
> that does
>   System.out.println(new File("d:\kafka-logs\test001-0\
> 00000000000000507600.index").delete())
> Does it work?
>
> One irritating thing about the delete API in Java is that it doesn't give
> an error so this can happen do to the file not existing or due to
> permissions and perhaps on Windows due to locking.
>
>
> On Wed, Dec 11, 2013 at 12:30 AM, CuiLiang <[EMAIL PROTECTED]> wrote:
>
> > Update this issue.
> > I update the log config to log.dirs=\\kafka-logs. The log file is deleted
> > but still can't delete the index file. I got below error message.
> >
> > [2013-12-11 00:07:59,671] INFO Deleting index
> > d:\kafka-logs\test001-0\00000000000000507600.index
> (kafka.log.OffsetIndex)
> > [2013-12-11 00:07:59,672] ERROR error in loggedRunnable
> > (kafka.utils.Utils$)
> > kafka.common.KafkaStorageException: Deleting log segment 507600 failed.
> > at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:613)
> > at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:608)
> > at
> >
> >
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
> > at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32)
> > at kafka.log.Log.deleteSegments(Log.scala:608)
> > at
> >
> >
> kafka.log.LogManager.kafka$log$LogManager$$cleanupSegmentsToMaintainSize(LogManager.scala:264)
> > at
> kafka.log.LogManager$$anonfun$cleanupLogs$2.apply(LogManager.scala:277)
> > at
> kafka.log.LogManager$$anonfun$cleanupLogs$2.apply(LogManager.scala:275)
> > at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> > at
> >
> >
> scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
> > at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
> > at
> >
> >
> scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:521)
> > at kafka.log.LogManager.cleanupLogs(LogManager.scala:275)
> > at
> >
> kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:141)
> > at kafka.utils.Utils$$anon$2.run(Utils.scala:68)
> > at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> > at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
> > at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> > Source)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> > Source)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> > at java.lang.Thread.run(Unknown Source)
> >
> > Thanks,
> > Liang Cui
> >
> >
> > 2013/12/11 CuiLiang <[EMAIL PROTECTED]>
> >
> > > The Kafka folder is d:\kafka_2.8.0-0.8.0. Kafka log folder is
> > > d:\data\kafka-logs. kafka batch folder is d:\kafka_2.8.0-0.8.0\bin.
> > > log.dirs=..\\..\\data\\kafka-logs
> > > The log segments files is created correct, but can't delete the log
> file.
> > > My machine has C, D, E,F, G, k partitions.
> > >
> > >
> > > Thanks,
> > > Liang Cui
> > >
> > >
> > > 2013/12/11 Jay Kreps <[EMAIL PROTECTED]>
> > >
> > >> What is your configuration for data.dirs (the path where data is) and
> > what
> > >> is the set of disks/volumes on the machine?
> > >>
> > >> -Jay
> > >>
> > >>
> > >> On Tue, Dec 10, 2013 at 12:50 AM, CuiLiang <[EMAIL PROTECTED]>
> > wrote:
> > >>
> > >> > Hi All,
> > >> >
> > >> > I'm use Kafka 0.8 release build with 1 partition, 1 replica. My OS
> is
> > >> > Windows server 2012 and JDK is 1.7. I got below error when Kafka

cuiliang
MSN : [EMAIL PROTECTED]