Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Migrating a cluster from 0.8.0 to 0.8.1


Copy link to this message
-
Re: Migrating a cluster from 0.8.0 to 0.8.1
Guozhang Wang 2013-12-24, 17:54
This OS should be similar to my desktop. Did you try

1. kafka-start-zookeeper
2. kafka-start-server
3. kafka-stop-server

And it does not work for you?

Guozhang
On Mon, Dec 23, 2013 at 2:50 PM, Drew Goya <[EMAIL PROTECTED]> wrote:

> We are running on an Amazon Linux AMI, this is our specific version:
>
> Linux version 2.6.32-220.23.1.el6.centos.plus.x86_64 (
> [EMAIL PROTECTED]) (gcc version 4.4.6 20110731 (Red Hat
> 4.4.6-3) (GCC) ) #1 SMP Tue Jun 19 04:14:37 BST 2012
>
>
> On Mon, Dec 23, 2013 at 11:24 AM, Guozhang Wang <[EMAIL PROTECTED]>
> wrote:
>
> > Hi Drew,
> >
> > I tried the kafka-server-stop script and it worked for me. Wondering
> which
> > OS are you using?
> >
> > Guozhang
> >
> >
> > On Mon, Dec 23, 2013 at 10:57 AM, Drew Goya <[EMAIL PROTECTED]> wrote:
> >
> > > Occasionally I do have to hard kill brokers, the kafka-server-stop.sh
> > > script stopped working for me a few months ago.  I saw another thread
> in
> > > the mailing list mentioning the issue too.  I'll change the signal back
> > to
> > > SIGTERM and run that way for a while, hopefully the problem goes away.
> > >
> > > This is the commit where it changed:
> > >
> > >
> > >
> >
> https://github.com/apache/kafka/commit/51de7c55d2b3107b79953f401fc8c9530bd0eea0
> > >
> > >
> > > On Mon, Dec 23, 2013 at 10:09 AM, Neha Narkhede <
> [EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Are you hard killing the brokers? And is this issue reproducible?
> > > >
> > > >
> > > > On Sat, Dec 21, 2013 at 11:39 AM, Drew Goya <[EMAIL PROTECTED]>
> > wrote:
> > > >
> > > > > Hey guys, another small issue to report for 0.8.1.  After a couple
> > > days 3
> > > > > of my brokers had fallen off the ISR list for a 2-3 of their
> > > partitions.
> > > > >
> > > > > I didn't see anything unusual in the log and I just restarted one.
> >  It
> > > > came
> > > > > up fine but as it loaded its logs I these messages showed up:
> > > > >
> > > > > [2013-12-21 19:25:19,968] WARN [ReplicaFetcherThread-0-2], Replica
> 1
> > > for
> > > > > partition [Events2,58] reset its fetch offset to current leader 2's
> > > start
> > > > > offset 1042738519 (kafka.server.ReplicaFetcherThread)
> > > > > [2013-12-21 19:25:19,969] WARN [ReplicaFetcherThread-0-14],
> Replica 1
> > > for
> > > > > partition [Events2,28] reset its fetch offset to current leader
> 14's
> > > > start
> > > > > offset 1043415514 (kafka.server.ReplicaFetcherThread)
> > > > > [2013-12-21 19:25:20,012] WARN [ReplicaFetcherThread-0-2], Current
> > > offset
> > > > > 1011209589 for partition [Events2,58] out of range; reset offset to
> > > > > 1042738519 (kafka.server.ReplicaFetcherThread)
> > > > > [2013-12-21 19:25:20,013] WARN [ReplicaFetcherThread-0-14], Current
> > > > offset
> > > > > 1010086751 for partition [Events2,28] out of range; reset offset to
> > > > > 1043415514 (kafka.server.ReplicaFetcherThread)
> > > > > [2013-12-21 19:25:20,036] WARN [ReplicaFetcherThread-0-14],
> Replica 1
> > > for
> > > > > partition [Events2,71] reset its fetch offset to current leader
> 14's
> > > > start
> > > > > offset 1026871415 (kafka.server.ReplicaFetcherThread)
> > > > > [2013-12-21 19:25:20,036] WARN [ReplicaFetcherThread-0-2], Replica
> 1
> > > for
> > > > > partition [Events2,44] reset its fetch offset to current leader 2's
> > > start
> > > > > offset 1052372907 (kafka.server.ReplicaFetcherThread)
> > > > > [2013-12-21 19:25:20,036] WARN [ReplicaFetcherThread-0-14], Current
> > > > offset
> > > > > 993879706 for partition [Events2,71] out of range; reset offset to
> > > > > 1026871415 (kafka.server.ReplicaFetcherThread)
> > > > > [2013-12-21 19:25:20,036] WARN [ReplicaFetcherThread-0-2], Current
> > > offset
> > > > > 1020715056 for partition [Events2,44] out of range; reset offset to
> > > > > 1052372907 (kafka.server.ReplicaFetcherThread)
> > > > >
> > > > > Judging by the network traffic and disk usage changes after the
> > reboot
> > > > > (both jumped up) a couple of the partition replicas had fallen