Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Connection reset by peer


+
Yonghui Zhao 2013-03-18, 03:25
+
Jun Rao 2013-03-18, 16:25
+
Yonghui Zhao 2013-03-19, 07:29
+
Yonghui Zhao 2013-03-19, 08:35
+
Jun Rao 2013-03-19, 16:15
+
Yonghui Zhao 2013-03-21, 04:26
+
Jun Rao 2013-03-22, 15:06
+
Yonghui Zhao 2013-03-22, 15:46
+
Yonghui Zhao 2013-03-25, 06:14
+
Neha Narkhede 2013-03-25, 13:49
+
Yonghui Zhao 2013-03-26, 02:14
+
Neha Narkhede 2013-03-26, 03:16
+
Yonghui Zhao 2013-03-26, 03:49
Copy link to this message
-
Re: Connection reset by peer
Did you have a gc pause around that time on the server ? What are your
server's current gc settings ?

Thanks,
Neha

On Mon, Mar 25, 2013 at 8:48 PM, Yonghui Zhao <[EMAIL PROTECTED]> wrote:
> Thanks Neha, btw have you seen this exception.  We didn't restart any
> service it happens in deep night.
>
> java.lang.RuntimeException: A broker is already registered on the path
> /brokers/ids/0. This probably indicates that you either have configured a
> brokerid that is already in use, or else you have shutdown this broker and
> restarted it faster than the zookeeper timeout so it appears to be
> re-registering.
>     at
> kafka.server.KafkaZooKeeper.registerBrokerInZk(KafkaZooKeeper.scala:57)
>     at
> kafka.server.KafkaZooKeeper$SessionExpireListener.handleNewSession(KafkaZooKeeper.scala:100)
>     at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472)
>     at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)
> [2013-03-26 02:07:19,155] INFO re-registering broker info in ZK for broker
> 0 (kafka.server.KafkaZooKeeper)
> [2013-03-26 02:07:19,155] INFO Registering broker /brokers/ids/0
> (kafka.server.KafkaZooKeeper)
> [2013-03-26 02:07:19,611] INFO conflict in /brokers/ids/0 data:
> 127.0.0.1-1364234839275:127.0.0.1:9093 stored data: 127.0.0.1-1364227372971:
> 127.0.0.1:9093 (kafka.utils.ZkUtils$)
> [2013-03-26 02:07:19,611] ERROR Error handling event ZkEvent[New session
> event sent to kafka.server.KafkaZooKeeper$SessionExpireListener@40f8c9bf]
> (org.I0Itec.zkclient.ZkEventThread)
> java.lang.RuntimeException: A broker is already registered on the path
> /brokers/ids/0. This probably indicates that you either have configured a
> brokerid that is already in use, or else you have shutdown this broker and
> restarted it faster than the zookeeper timeout so it appears to be
> re-registering.
>     at
> kafka.server.KafkaZooKeeper.registerBrokerInZk(KafkaZooKeeper.scala:57)
>     at
> kafka.server.KafkaZooKeeper$SessionExpireListener.handleNewSession(KafkaZooKeeper.scala:100)
>     at org.I0Itec.zkclient.ZkClient$4.run(ZkClient.java:472)
>     at org.I0Itec.zkclient.ZkEventThread.run(ZkEventThread.java:71)
>
>
>
> 2013/3/26 Neha Narkhede <[EMAIL PROTECTED]>
>
>> That really depends on your consumer application's memory allocation
>> patterns. If it is a thin wrapper over a Kafka consumer, I would imagine
>> you can get away with using CMS for the tenured generation and parallel
>> collector for the new generation with a small heap like 1gb or so.
>>
>> Thanks,
>> Neha
>>
>> On Monday, March 25, 2013, Yonghui Zhao wrote:
>>
>> > Any suggestion on consumer side?
>> > 在 2013-3-25 下午9:49,"Neha Narkhede" <[EMAIL PROTECTED]<javascript:;>
>> > >写道:
>> >
>> > > For Kafka 0.7 in production at Linkedin, we use a heap of size 3G, new
>> > gen
>> > > 256 MB, CMS collector with occupancy of 70%.
>> > >
>> > > Thanks,
>> > > Neha
>> > >
>> > > On Sunday, March 24, 2013, Yonghui Zhao wrote:
>> > >
>> > > > Hi Jun,
>> > > >
>> > > > I used kafka-server-start.sh to start kafka, there is only one jvm
>> > > setting
>> > > > "-Xmx512M“
>> > > >
>> > > > Do you have some recommend GC setting?   Usually our sever has 32GB
>> or
>> > > 64GB
>> > > > RAM.
>> > > >
>> > > > 2013/3/22 Jun Rao <[EMAIL PROTECTED]>
>> > > >
>> > > > > A typical reason for many rebalancing is the consumer side GC. If
>> so,
>> > > you
>> > > > > will see logs in the consume saying sth like "expired session" for
>> > ZK.
>> > > > > Occasional rebalances are fine. Too many rebalances can slow down
>> the
>> > > > > consumption and you will need to tune your GC setting.
>> > > > >
>> > > > > Thanks,
>> > > > >
>> > > > > Jun
>> > > > >
>> > > > > On Thu, Mar 21, 2013 at 11:07 PM, Yonghui Zhao <
>> > [EMAIL PROTECTED]
>> > > > > >wrote:
>> > > > >
>> > > > > > Yes, before consumer exception:
>> > > > > >
>> > > > > > 2013/03/21 12:07:17.909 INFO [ZookeeperConsumerConnector] []
>> > > > > > 0_lg-mc-db01.bj-1363784482043-f98c7868 *end rebalancing
>> > > > > > consumer*0_lg-mc-db01.bj-1363784482043-f98c7868 try #0

 
+
Yonghui Zhao 2013-03-26, 16:48
+
Neha Narkhede 2013-03-27, 14:52
+
Yonghui Zhao 2013-03-28, 03:35
+
Jun Rao 2013-03-28, 04:54
+
Yonghui Zhao 2013-03-28, 07:23
+
Jun Rao 2013-03-28, 14:52
+
Yonghui Zhao 2013-03-28, 15:21
+
Jun Rao 2013-03-28, 15:25
+
Yonghui Zhao 2013-03-28, 15:32
+
Jun Rao 2013-03-29, 04:03
+
Ranjith Venkatesan 2014-02-02, 06:03