Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Re: upgrade from beta1 to 0.81


Copy link to this message
-
Re: upgrade from beta1 to 0.81
Guozhang Wang 2013-12-24, 18:53
This does include the patch of 1112, which means the issue is not fixed.

Drew, could you comment on KAFKA-1112 about how you can reproduce this
issue so we can re-open it?

Guozhang
On Mon, Dec 23, 2013 at 11:03 AM, Drew Goya <[EMAIL PROTECTED]> wrote:

> I'm not sure if I had hard killed the broker but I do have the fix for that
> case.
>
> I currently have this commit deployed:
>
> commit 87efda7f818218e0868be7032c73c994d75931fd
> Author: Guozhang Wang <[EMAIL PROTECTED]>
> Date:   Fri Nov 22 09:16:39 2013 -0800
>
>     kafka-1103; Consumer uses two zkclients; patched by Guozhang Wang;
> reviewed by Joel Koshy and Jun Rao
>
>
> On Mon, Dec 23, 2013 at 9:59 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Did you hard kill the broker?  If so, do you have the fix for KAFKA-1112?
> >
> > Thanks,
> >
> > Jun
> >
> >
> > On Fri, Dec 20, 2013 at 4:05 PM, Drew Goya <[EMAIL PROTECTED]> wrote:
> >
> > > This is the exception I ran into, I was able to fix it by deleting the
> > > /data/kafka/logs/Events2-124/ directory.  That directory contained a
> non
> > > zero size index file and a zero size log file.  I had a bunch of these
> > > directories scattered around the cluster.
> > >
> > > [2013-12-18 02:40:37,163] FATAL Fatal error during KafkaServerStable
> > > startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
> > > java.lang.IllegalArgumentException: requirement failed: Corrupt index
> > > found, index file
> > (/data/kafka/logs/Events2-124/00000000000000000000.index)
> > > has non-zero size but the last offset is 0 and the base offset is 0
> > > at scala.Predef$.require(Predef.scala:145)
> > > at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:160)
> > > at kafka.log.Log$$anonfun$loadSegments$5.apply(Log.scala:159)
> > > at scala.collection.Iterator$class.foreach(Iterator.scala:631)
> > > at
> > >
> > >
> >
> scala.collection.JavaConversions$JIteratorWrapper.foreach(JavaConversions.scala:474)
> > > at scala.collection.IterableLike$class.foreach(IterableLike.scala:79)
> > > at
> > >
> > >
> >
> scala.collection.JavaConversions$JCollectionWrapper.foreach(JavaConversions.scala:495)
> > > at kafka.log.Log.loadSegments(Log.scala:159)
> > > at kafka.log.Log.<init>(Log.scala:64)
> > > at
> > >
> > >
> >
> kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$3.apply(LogManager.scala:120)
> > > at
> > >
> > >
> >
> kafka.log.LogManager$$anonfun$loadLogs$1$$anonfun$apply$3.apply(LogManager.scala:115)
> > > at
> > >
> > >
> >
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
> > > at scala.collection.mutable.ArrayOps.foreach(ArrayOps.scala:34)
> > > at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:115)
> > > at kafka.log.LogManager$$anonfun$loadLogs$1.apply(LogManager.scala:107)
> > > at
> > >
> > >
> >
> scala.collection.IndexedSeqOptimized$class.foreach(IndexedSeqOptimized.scala:34)
> > > at scala.collection.mutable.WrappedArray.foreach(WrappedArray.scala:32)
> > > at kafka.log.LogManager.loadLogs(LogManager.scala:107)
> > > at kafka.log.LogManager.<init>(LogManager.scala:59)
> > >
> > >
> > > On Fri, Dec 20, 2013 at 9:06 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
> > >
> > > > Drew,
> > > >
> > > > Even without kafka-1074, brokers shouldn't hit exceptions during
> > startup.
> > > > What exceptions do you see?
> > > >
> > > > Thanks,
> > > >
> > > > Jun
> > > >
> > > >
> > > > On Thu, Dec 19, 2013 at 9:36 PM, Drew Goya <[EMAIL PROTECTED]>
> wrote:
> > > >
> > > > > We migrated from 0.8.0 to 0.8.1 last week.  We have a 15 broker
> > cluster
> > > > so
> > > > > it took a while to roll through them one by one.  Once I finished I
> > was
> > > > > finally able to complete a partition reassignment.  I also had to
> do
> > > some
> > > > > manual cleanup, but Neha says it will be fixed soon:
> > > > >
> > > > > https://issues.apache.org/jira/browse/KAFKA-1074
> > > > >
> > > > > Until then, if you have done any partition reassignment you will
> have
> > > to
> > > > > watch your brokers as they come up.  They may fail and you will