Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> uncontrolled shutdown


Copy link to this message
-
Re: uncontrolled shutdown
Hello, yes, I can easily reproduce it, will send it to you asap.

Thanks.
On Fri, Nov 1, 2013 at 2:05 PM, Guozhang Wang <[EMAIL PROTECTED]> wrote:

> Hello Kane, do you mind attach your index file causing the issue if you
> still have it?
>
> Guozhang
>
>
> On Fri, Nov 1, 2013 at 8:42 AM, Guozhang Wang <[EMAIL PROTECTED]> wrote:
>
> > Currently the index files will only be deleted on startup if there are
> any
> > .swap file indicating the server crashed while opening the log segments.
> We
> > should probably change this logic.
> >
> > Guozhang
> >
> >
> > On Fri, Nov 1, 2013 at 8:16 AM, Kane Kane <[EMAIL PROTECTED]> wrote:
> >
> >> Filed a bug meanwhile, https://issues.apache.org/jira/browse/KAFKA-1112
> >>
> >>
> >> On Fri, Nov 1, 2013 at 8:15 AM, Neha Narkhede <[EMAIL PROTECTED]
> >> >wrote:
> >>
> >> > Jun,
> >> >
> >> > I've seen this happen in the latest code in trunk, need to remember
> how
> >> to
> >> > reproduce it.
> >> >
> >> > Thanks,
> >> > Neha
> >> > On Nov 1, 2013 7:59 AM, "Jun Rao" <[EMAIL PROTECTED]> wrote:
> >> >
> >> > > Are you using the latest code in the 0.8 branch?
> >> > >
> >> > > Thanks,
> >> > >
> >> > > Jun
> >> > >
> >> > >
> >> > > On Fri, Nov 1, 2013 at 7:36 AM, Kane Kane <[EMAIL PROTECTED]>
> >> wrote:
> >> > >
> >> > > > Neha, yes when i kill it with -9, sure I will file a bug.
> >> > > >
> >> > > > Thanks.
> >> > > >
> >> > > >
> >> > > > On Fri, Nov 1, 2013 at 3:43 AM, Neha Narkhede <
> >> [EMAIL PROTECTED]
> >> > > > >wrote:
> >> > > >
> >> > > > > When you say the Kafka dies, did you mean a kill -9 or kill -15
> ?
> >> I
> >> > > agree
> >> > > > > that Kafka should try and delete/rebuild index files, but in
> this
> >> > case,
> >> > > > it
> >> > > > > sounds like a bug. Can you file a JIRA?
> >> > > > >
> >> > > > > Thanks,
> >> > > > > Neha
> >> > > > >
> >> > > > >
> >> > > > > On Fri, Nov 1, 2013 at 1:51 AM, Kane Kane <
> [EMAIL PROTECTED]>
> >> > > wrote:
> >> > > > >
> >> > > > > > When machine with kafka dies, most often broker cannot start
> >> itself
> >> > > > with
> >> > > > > > errors about index files being corrupt. After i delete them
> >> > manually
> >> > > > > broker
> >> > > > > > usually can boot up successfully. Shouldn't kafka try to
> >> > > delete/rebuild
> >> > > > > > broken index files itself?
> >> > > > > >
> >> > > > > > Also this exception looks a bit weird:
> >> > > > > > java.lang.IllegalArgumentException: requirement failed:
> Corrupt
> >> > index
> >> > > > > > found, index file
> >> > > > > > (/disk1/kafka-logs/benchmark-2/00000000000000385258.index) has
> >> > > non-zero
> >> > > > > > size but the last offset is 385258 and the base offset is
> 385258
> >> > > > > >
> >> > > > > > Thanks.
> >> > > > > >
> >> > > > >
> >> > > >
> >> > >
> >> >
> >>
> >
> >
> >
> > --
> > -- Guozhang
> >
>
>
>
> --
> -- Guozhang
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB