Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Connection reset by peer


Copy link to this message
-
Re: Connection reset by peer
Yonghui Zhao 2013-03-28, 07:23
I used zookeeper-3.3.4 in kafka.

Default tickTime is 3 seconds, minSesstionTimeOut is 6 seconds.
Now I change tickTime to 5 seconds. minSesstionTimeOut to 10 seconds
But if we change timeout to a larger one,
"you have shutdown this broker and restarted it faster than the zookeeper
timeout so it appears to be re-registering."
this could happened more easily

Do you think consumer GC will affect kafka server and zk connection?

2013/3/28 Jun Rao <[EMAIL PROTECTED]>

> Not sure why the re-registration fails. Are you using ZK 3.3.4 or above?
>
> It seems that you consumer still GCs, which is the root cause. So, you will
> need to tune the GC setting further. Another way to avoid ZK session
> timeout is to increase the session timeout config.
>
> Thanks,
>
> Jun
>
> On Wed, Mar 27, 2013 at 8:35 PM, Yonghui Zhao <[EMAIL PROTECTED]>
> wrote:
>
> > Now I used GC like this:
> >
> > -server -Xms1536m -Xmx1536m -XX:NewSize=128m -XX:MaxNewSize=128m
> > -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
> > -XX:CMSInitiatingOccupancyFraction=70
> >
> >
> > But it still happened.  It seems kafka server reconnect with zk, but the
> > old node was still there. So kafka server stopped.
> > Can kafka server retry to connect with zk?
> >
> >
> > 2013-03-27 22:15:03,529] INFO Opening socket connection to server
> > localhost/
> > 127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
> > [2013-03-27 22:15:03,529] INFO Socket connection established to
> localhost/
> > 127.0.0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
> > [2013-03-27 22:15:05,855] INFO Session establishment complete on server
> > localhost/127.0.0.1:2181, sessionid = 0x13da6d94abf00aa, negotiated
> > timeout
> > = 6000 (org.apache.zookeeper.ClientCnxn)
> > [2013-03-27 22:15:05,942] INFO zookeeper state changed (SyncConnected)
> > (org.I0Itec.zkclient.ZkClient)
> > [2013-03-27 22:15:14,912] INFO conflict in /brokers/ids/0 data:
> > 127.0.0.1-1364393691770:127.0.0.1:9093 stored data: null
> > (kafka.utils.ZkUtils$)
> > [2013-03-27 22:15:14,942] ERROR Error handling event ZkEvent[New session
> > event sent to kafka.server.KafkaZooKeeper$SessionExpireListener@18f389bc
> ]
> > (org.I0Itec.zkclient.ZkEventThread)
> > java.lang.RuntimeException: A broker is already registered on the path
> > /brokers/ids/0. This probably indicates that you either have configured a
> > brokerid that is already in use, or else you have shutdown this broker
> and
> > restarted it faster than the zookeeper timeout so it appears to be
> > re-registering.
> >     at
> > kafka.server.KafkaZooKeeper.registerBrokerInZk(KafkaZooKeeper.scala:57)
> >     at
> >
> >
> kafka.server.KafkaZooKeeper$SessionExpireListener.handleNewSession(KafkaZooKeeper.scala:100)
> >     at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472)
> >     at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)
> > [2013-03-27 22:15:33,736] INFO Closing socket connection to /127.0.0.1.
> > (kafka.network.Processor)
> >
> >
> >
> >
> >
> > 2013/3/27 Neha Narkhede <[EMAIL PROTECTED]>
> >
> > > The kafka-server-start.sh script doesn't have the mentioned GC
> > > settings and heap size configured. However, probably doing that is a
> > > good idea.
> > >
> > > Thanks,
> > > Neha
> > >
> > > On Tue, Mar 26, 2013 at 9:47 AM, Yonghui Zhao <[EMAIL PROTECTED]>
> > > wrote:
> > > > kafka server is started by bin/kafka-server-start.sh.  No gc setting.
> > > > 在 2013-3-26 下午11:40,"Neha Narkhede" <[EMAIL PROTECTED]>写道:
> > > >
> > > >> Did you have a gc pause around that time on the server ? What are
> your
> > > >> server's current gc settings ?
> > > >>
> > > >> Thanks,
> > > >> Neha
> > > >>
> > > >> On Mon, Mar 25, 2013 at 8:48 PM, Yonghui Zhao <
> [EMAIL PROTECTED]>
> > > >> wrote:
> > > >> > Thanks Neha, btw have you seen this exception.  We didn't restart
> > any
> > > >> > service it happens in deep night.
> > > >> >
> > > >> > java.lang.RuntimeException: A broker is already registered on the
> > path
> > > >> > /brokers/ids/0. This probably indicates that you either have