Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> RE: Looks like consumer fetchers get stopped we are not getting any data


Copy link to this message
-
Re: Looks like consumer fetchers get stopped we are not getting any data
Have you looked at our FAQ, especially
https://cwiki.apache.org/confluence/display/KAFKA/FAQ#FAQ-Whyaretheremanyrebalancesinmyconsumerlog
?

Thanks,

Jun
On Fri, Jan 10, 2014 at 2:25 PM, Seshadri, Balaji
<[EMAIL PROTECTED]>wrote:

> Any clue would be helpful.
>
> -----Original Message-----
> From: Seshadri, Balaji [mailto:[EMAIL PROTECTED]]
> Sent: Friday, January 10, 2014 12:46 PM
> To: [EMAIL PROTECTED]
> Subject: RE: Looks like consumer fetchers get stopped we are not getting
> any data
>
> Yes rebalance begins and exceptions occurs.
>
>
> {2014-01-10 00:58:11,293} INFO
>  [account-info-updated-hadoop-consumer_tm1mwdpl04-1389222553159-ad59660b_watcher_executor]
> (?:?) - [account-i
> nfo-updated-hadoop-consumer_tm1mwdpl04-1389222553159-ad59660b], Cleared the
> data chunks in all the consumer message iterators
> {2014-01-10 00:58:11,293} INFO
>  [account-info-updated-hadoop-consumer_tm1mwdpl04-1389222553159-ad59660b_watcher_executor]
> (?:?) - [account-i
> nfo-updated-hadoop-consumer_tm1mwdpl04-1389222553159-ad59660b], Committing
> all offsets after clearing the fetcher queues
> {2014-01-10 00:58:11,298} DEBUG [catalina-exec-12-SendThread(
> tvip-m1-mw-zookeeper.dish.com:2181)] (ClientCnxn.java:759) - Got ping
> response for sessionid: 0x1437b2879870005 af ter 0ms
> {2014-01-10 00:58:11,313} INFO
>  [account-activated-hadoop-consumer_tm1mwdpl04-1389222557906-562b6738_watcher_executor]
> (?:?) -
> [account-activated-hadoop-consumer_tm1mwdpl04-1389222557906-562b6738],
> begin rebalancing consumer
> account-activated-hadoop-consumer_tm1mwdpl04-1389222557906-562b6738 try #1
> {2014-01-10 00:58:11,314} DEBUG [catalina-exec-12-SendThread(
> tvip-m1-mw-zookeeper.dish.com:2181)] (ClientCnxn.java:839) - Reading
> reply sessionid:0x1437b2879870005, packet:: clientPath:null serverPath:null
> finished:false header:: 627,8  replyHeader:: 627,51539619966,0  request::
> '/brokers/ids,F  response:: v{'1}
> {2014-01-10 00:58:11,315} DEBUG [catalina-exec-12-SendThread(
> tvip-m1-mw-zookeeper.dish.com:2181)] (ClientCnxn.java:839) - Reading
> reply sessionid:0x1437b2879870005, packet:: clientPath:null serverPath:null
> finished:false header:: 628,4  replyHeader:: 628,51539619966,0  request::
> '/brokers/ids/1,F  response::
> #7b2022686f7374223a22746d312d6b61666b6162726f6b6572313031222c20226a6d785f706f7274223a393939392c2022706f7274223a393039322c202276657273696f6e223a31207d,s{47244644685,47244644685,1388537628753,1388537628753,0,0,0,163056791896588316,74,0,47244644685}
> {2014-01-10 00:58:11,316} DEBUG [catalina-exec-12-SendThread(
> tvip-m1-mw-zookeeper.dish.com:2181)] (ClientCnxn.java:839) - Reading
> reply sessionid:0x1437b2879870005, packet:: clientPath:null serverPath:null
> finished:false header:: 629,4  replyHeader:: 629,51539619966,-101
>  request::
> '/consumers/account-activated-hadoop-consumer/ids/account-activated-hadoop-consumer_tm1mwdpl04-1389222557906-562b6738,F
>  response::
> {2014-01-10 00:58:11,316} INFO
>  [account-activated-hadoop-consumer_tm1mwdpl04-1389222557906-562b6738_watcher_executor]
> (?:?) -
> [account-activated-hadoop-consumer_tm1mwdpl04-1389222557906-562b6738],
> exception during rebalance
> org.I0Itec.zkclient.exception.ZkNoNodeException:
> org.apache.zookeeper.KeeperException$NoNodeException: KeeperErrorCode =
> NoNode for
> /consumers/account-activated-hadoop-consumer/ids/account-activated-hadoop-consumer_tm1mwdpl04-1389222557906-562b6738
>         at
> org.I0Itec.zkclient.exception.ZkException.create(ZkException.java:47)
>         at
> org.I0Itec.zkclient.ZkClient.retryUntilConnected(ZkClient.java:685)
>         at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:766)
>         at org.I0Itec.zkclient.ZkClient.readData(ZkClient.java:761)
>         at kafka.utils.ZkUtils$.readData(Unknown Source)
>         at kafka.consumer.TopicCount$.constructTopicCount(Unknown Source)
>         at
> kafka.consumer.ZookeeperConsumerConnector$ZKRebalancerListener.kafka$consumer$ZookeeperConsumerConnector$ZKRebalancerListener$$rebalance(Unknown