Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka 0.8 delete log failed


Copy link to this message
-
Re: Kafka 0.8 delete log failed
I tried the sample code and it works. I also can delete the old index file
manually.

Thanks,
Liang Cui
2013/12/12 Jay Kreps <[EMAIL PROTECTED]>

> Is the path d:\kafka-logs\test001-0\00000000000000507600.index correct?
>
> The tricky thing here is we don't have access to windows for testing so we
> will need a bit more help for debugging. If you write a simple Java program
> that does
>   System.out.println(new File("d:\kafka-logs\test001-0\
> 00000000000000507600.index").delete())
> Does it work?
>
> One irritating thing about the delete API in Java is that it doesn't give
> an error so this can happen do to the file not existing or due to
> permissions and perhaps on Windows due to locking.
>
>
> On Wed, Dec 11, 2013 at 12:30 AM, CuiLiang <[EMAIL PROTECTED]> wrote:
>
> > Update this issue.
> > I update the log config to log.dirs=\\kafka-logs. The log file is deleted
> > but still can't delete the index file. I got below error message.
> >
> > [2013-12-11 00:07:59,671] INFO Deleting index
> > d:\kafka-logs\test001-0\00000000000000507600.index
> (kafka.log.OffsetIndex)
> > [2013-12-11 00:07:59,672] ERROR error in loggedRunnable
> > (kafka.utils.Utils$)
> > kafka.common.KafkaStorageException: Deleting log segment 507600 failed.
> > at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:613)
> > at kafka.log.Log$$anonfun$deleteSegments$1.apply(Log.scala:608)
> > at
> >
> >
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
> > at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32)
> > at kafka.log.Log.deleteSegments(Log.scala:608)
> > at
> >
> >
> kafka.log.LogManager.kafka$log$LogManager$$cleanupSegmentsToMaintainSize(LogManager.scala:264)
> > at
> kafka.log.LogManager$$anonfun$cleanupLogs$2.apply(LogManager.scala:277)
> > at
> kafka.log.LogManager$$anonfun$cleanupLogs$2.apply(LogManager.scala:275)
> > at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> > at
> >
> >
> scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
> > at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
> > at
> >
> >
> scala.collection.JavaConversions$JListWrapper.foreach(JavaConversions.scala:521)
> > at kafka.log.LogManager.cleanupLogs(LogManager.scala:275)
> > at
> >
> kafka.log.LogManager$$anonfun$startup$1.apply$mcV$sp(LogManager.scala:141)
> > at kafka.utils.Utils$$anon$2.run(Utils.scala:68)
> > at java.util.concurrent.Executors$RunnableAdapter.call(Unknown Source)
> > at java.util.concurrent.FutureTask$Sync.innerRunAndReset(Unknown Source)
> > at java.util.concurrent.FutureTask.runAndReset(Unknown Source)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(Unknown
> > Source)
> > at
> >
> >
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(Unknown
> > Source)
> > at java.util.concurrent.ThreadPoolExecutor.runWorker(Unknown Source)
> > at java.util.concurrent.ThreadPoolExecutor$Worker.run(Unknown Source)
> > at java.lang.Thread.run(Unknown Source)
> >
> > Thanks,
> > Liang Cui
> >
> >
> > 2013/12/11 CuiLiang <[EMAIL PROTECTED]>
> >
> > > The Kafka folder is d:\kafka_2.8.0-0.8.0. Kafka log folder is
> > > d:\data\kafka-logs. kafka batch folder is d:\kafka_2.8.0-0.8.0\bin.
> > > log.dirs=..\\..\\data\\kafka-logs
> > > The log segments files is created correct, but can't delete the log
> file.
> > > My machine has C, D, E,F, G, k partitions.
> > >
> > >
> > > Thanks,
> > > Liang Cui
> > >
> > >
> > > 2013/12/11 Jay Kreps <[EMAIL PROTECTED]>
> > >
> > >> What is your configuration for data.dirs (the path where data is) and
> > what
> > >> is the set of disks/volumes on the machine?
> > >>
> > >> -Jay
> > >>
> > >>
> > >> On Tue, Dec 10, 2013 at 12:50 AM, CuiLiang <[EMAIL PROTECTED]>
> > wrote:
> > >>
> > >> > Hi All,
> > >> >
> > >> > I'm use Kafka 0.8 release build with 1 partition, 1 replica. My OS
> is
> > >> > Windows server 2012 and JDK is 1.7. I got below error when Kafka

cuiliang
MSN : [EMAIL PROTECTED]

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB