Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka issue with "Reconnect due to socket error"


Copy link to this message
-
Re: Kafka issue with "Reconnect due to socket error"
The reconnect is due to the socket connection to Kafka broker. Technically,
reconnect worked. So the tool still produces the output. The reconnection
is weird though. Do you see the same reconnection when you run console
consumer?

Thanks,

Jun
On Tue, Jul 9, 2013 at 11:06 AM, Dennis Haller <[EMAIL PROTECTED]>wrote:

> Hi,
>
> Thanks for your reply.
>
> I can't see anything in the broker logs, , except this keeps coming up in
> the zookeeper log (instance 1 out of 3):
> WARN  [SyncThread:1:FileTxnLog@321] - fsync-ing the write ahead log in
> SyncThread:1 took 1298ms which will adversely effect operation latency. See
> the ZooKeeper troubleshooting guide
>
>
> I'm not sure if the connection error is from zookeeper or a kafka broker.
> In an edited version of the log file, you see that a normal cycle of
> ConsmerOffsetChecker requires 5 ZK queries before the stats are printed
> out, but when the Exception is thrown it is after 4 queries have completed
> and somewhere in the 5th query:
>
>
> [2013-07-09 10:45:41,845] INFO Client
> environment:user.dir=/opt/kafka/kafka-0.8 (org.apache.zookeeper.ZooKeeper)
>  18 [2013-07-09 10:45:41,846] INFO Initiating client connection,
> connectString=ec2-23-22-34-191.compute-1.amazonaws.comsessionTimeout=30000
> watcher=org.I0Itec.zkclient.ZkClient@3c0db454 (>
>  19 [2013-07-09 10:45:41,848] DEBUG zookeeper.disableAutoWatchReset is
> false (org.apache.zookeeper.ClientCnxn)
>  20 [2013-07-09 10:45:41,934] DEBUG Awaiting connection to Zookeeper server
> (org.I0Itec.zkclient.ZkClient)
>  21 [2013-07-09 10:45:41,934] DEBUG Waiting for keeper state SyncConnected
> (org.I0Itec.zkclient.ZkClient)
>  22 [2013-07-09 10:45:41,934] INFO Opening socket connection to server
>
> ec2-23-22-34-191.compute-1.amazonaws.com/23.22.34.191:2181(org.apache.zookeeper.ClientCnxn)
>  23 [2013-07-09 10:45:42,020] INFO Socket connection established to
> ec2-23-22-34-191.compute-1.amazonaws.com/23.22.34.191:2181, initiating
> session (org.apache.zookeeper.ClientCnxn)
>  24 [2013-07-09 10:45:42,024] DEBUG Session establishment request sent on
>
> ec2-23-22-34-191.compute-1.amazonaws.com/23.22.34.191:2181(org.apache.zookeeper.ClientCnxn)
>  25 [2013-07-09 10:45:42,114] INFO Session establishment complete on server
> ec2-23-22-34-191.compute-1.amazonaws.com/23.22.34.191:2181, sessionid =
> 0x13fc464f0420009, negotiated timeout = >
>  26 [2013-07-09 10:45:42,118] DEBUG Received event: WatchedEvent
> state:SyncConnected type:None path:null (org.I0Itec.zkclient.ZkClient)
>  27 [2013-07-09 10:45:42,118] INFO zookeeper state changed (SyncConnected)
> (org.I0Itec.zkclient.ZkClient)
>  28 [2013-07-09 10:45:42,118] DEBUG Leaving process event
> (org.I0Itec.zkclient.ZkClient)
>  29 [2013-07-09 10:45:42,118] DEBUG State is SyncConnected
> (org.I0Itec.zkclient.ZkClient)
>  30 [2013-07-09 10:45:42,265] DEBUG Reading reply
> sessionid:0x13fc464f0420009, packet:: clientPath:null serverPath:null
> finished:false header:: 1,8  replyHeader:: 1,12884971368,0  request:>
>  31 Group           Topic                          Pid Offset
>  logSize         Lag             Owner
>  32
>  33 [2013-07-09 10:45:42,394] DEBUG Reading reply
> sessionid:0x13fc464f0420009, ..........request::
> '/brokers/topics/qa-M-Candidate-CrmStage-Events,F
>  (org.apache.zookeeper.ClientCnxn)
>  34 [2013-07-09 10:45:42,684] DEBUG Reading reply
> sessionid:0x13fc464f0420009, ..........request::
> '/consumers/KafkaMirror/offsets/qa-M-Candidate-CrmStage-Events/0,F
>  (org.apache.zookeeper>
>  35 [2013-07-09 10:45:42,768] DEBUG Reading reply
> sessionid:0x13fc464f0420009, ..........request::
> '/consumers/KafkaMirror/owners/qa-M-Candidate-CrmStage-Events/0,F
>  (org.apache.zookeeper.>
>  36 [2013-07-09 10:45:42,849] DEBUG Reading reply
> sessionid:0x13fc464f0420009, ..........request::
> '/brokers/topics/qa-M-Candidate-CrmStage-Events/partitions/0/state,F
>  (org.apache.zookeep>
>  37 [2013-07-09 10:45:42,971] DEBUG Reading reply
> sessionid:0x13fc464f0420009, ..........request:: '/brokers/ids/2,F