Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Connection reset by peer


Copy link to this message
-
Re: Connection reset by peer
Jun Rao 2013-03-28, 04:54
Not sure why the re-registration fails. Are you using ZK 3.3.4 or above?

It seems that you consumer still GCs, which is the root cause. So, you will
need to tune the GC setting further. Another way to avoid ZK session
timeout is to increase the session timeout config.

Thanks,

Jun

On Wed, Mar 27, 2013 at 8:35 PM, Yonghui Zhao <[EMAIL PROTECTED]> wrote:

> Now I used GC like this:
>
> -server -Xms1536m -Xmx1536m -XX:NewSize=128m -XX:MaxNewSize=128m
> -XX:+UseConcMarkSweepGC -XX:+UseParNewGC
> -XX:CMSInitiatingOccupancyFraction=70
>
>
> But it still happened.  It seems kafka server reconnect with zk, but the
> old node was still there. So kafka server stopped.
> Can kafka server retry to connect with zk?
>
>
> 2013-03-27 22:15:03,529] INFO Opening socket connection to server
> localhost/
> 127.0.0.1:2181 (org.apache.zookeeper.ClientCnxn)
> [2013-03-27 22:15:03,529] INFO Socket connection established to localhost/
> 127.0.0.1:2181, initiating session (org.apache.zookeeper.ClientCnxn)
> [2013-03-27 22:15:05,855] INFO Session establishment complete on server
> localhost/127.0.0.1:2181, sessionid = 0x13da6d94abf00aa, negotiated
> timeout
> = 6000 (org.apache.zookeeper.ClientCnxn)
> [2013-03-27 22:15:05,942] INFO zookeeper state changed (SyncConnected)
> (org.I0Itec.zkclient.ZkClient)
> [2013-03-27 22:15:14,912] INFO conflict in /brokers/ids/0 data:
> 127.0.0.1-1364393691770:127.0.0.1:9093 stored data: null
> (kafka.utils.ZkUtils$)
> [2013-03-27 22:15:14,942] ERROR Error handling event ZkEvent[New session
> event sent to kafka.server.KafkaZooKeeper$SessionExpireListener@18f389bc]
> (org.I0Itec.zkclient.ZkEventThread)
> java.lang.RuntimeException: A broker is already registered on the path
> /brokers/ids/0. This probably indicates that you either have configured a
> brokerid that is already in use, or else you have shutdown this broker and
> restarted it faster than the zookeeper timeout so it appears to be
> re-registering.
>     at
> kafka.server.KafkaZooKeeper.registerBrokerInZk(KafkaZooKeeper.scala:57)
>     at
>
> kafka.server.KafkaZooKeeper$SessionExpireListener.handleNewSession(KafkaZooKeeper.scala:100)
>     at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472)
>     at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)
> [2013-03-27 22:15:33,736] INFO Closing socket connection to /127.0.0.1.
> (kafka.network.Processor)
>
>
>
>
>
> 2013/3/27 Neha Narkhede <[EMAIL PROTECTED]>
>
> > The kafka-server-start.sh script doesn't have the mentioned GC
> > settings and heap size configured. However, probably doing that is a
> > good idea.
> >
> > Thanks,
> > Neha
> >
> > On Tue, Mar 26, 2013 at 9:47 AM, Yonghui Zhao <[EMAIL PROTECTED]>
> > wrote:
> > > kafka server is started by bin/kafka-server-start.sh.  No gc setting.
> > > 在 2013-3-26 下午11:40,"Neha Narkhede" <[EMAIL PROTECTED]>写道:
> > >
> > >> Did you have a gc pause around that time on the server ? What are your
> > >> server's current gc settings ?
> > >>
> > >> Thanks,
> > >> Neha
> > >>
> > >> On Mon, Mar 25, 2013 at 8:48 PM, Yonghui Zhao <[EMAIL PROTECTED]>
> > >> wrote:
> > >> > Thanks Neha, btw have you seen this exception.  We didn't restart
> any
> > >> > service it happens in deep night.
> > >> >
> > >> > java.lang.RuntimeException: A broker is already registered on the
> path
> > >> > /brokers/ids/0. This probably indicates that you either have
> > configured a
> > >> > brokerid that is already in use, or else you have shutdown this
> broker
> > >> and
> > >> > restarted it faster than the zookeeper timeout so it appears to be
> > >> > re-registering.
> > >> >     at
> > >> >
> > kafka.server.KafkaZooKeeper.registerBrokerInZk(KafkaZooKeeper.scala:57)
> > >> >     at
> > >> >
> > >>
> >
> kafka.server.KafkaZooKeeper$SessionExpireListener.handleNewSession(KafkaZooKeeper.scala:100)
> > >> >     at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472)
> > >> >     at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)
> > >> > [2013-03-26 02:07:19,155] INFO re-registering broker info in ZK for