Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Controlled shutdown failure, retry settings


Copy link to this message
-
Controlled shutdown failure, retry settings
I'm running into an issue where sometimes, the controlled shutdown fails to
complete after the default 3 retry attempts.  This ended up in one case,
with a broker under going an unclean shutdown, and then it was in a rather
bad state after restart.  Producers would connect to the metadata vip,
still think that this broker was the leader, and then fail on that leader,
and then reconnect to to the metadata vip, and get sent back to that same
failed broker!   Does that make sense?

I'm trying to understand the conditions which cause the controlled shutdown
to fail?  There doesn't seem to be a setting for max amount of time to
wait, etc.

It would be nice to specify how long to try before giving up (hopefully
giving up in a graceful way).

Instead, we have a retry count, but it's not clear what this retry count is
really specifying, in terms of how long to keep trying, etc.

Also, what are the ramifications for different settings for the
controlled.shutdown.retry.backoff.ms?  Is there a reason we want to wait
before retrying again (again, it would be helpful to understand the reasons
for a controlled shutdown failure).

Thanks,

Jason

 
+
Joel Koshy 2013-10-25, 14:35
+
Neha Narkhede 2013-10-25, 14:41
+
Jason Rosenberg 2013-10-25, 15:26
+
Neha Narkhede 2013-10-25, 16:39
+
Jason Rosenberg 2013-10-25, 16:51
+
Jason Rosenberg 2013-10-25, 22:22
+
Joel Koshy 2013-10-26, 01:17
+
Jason Rosenberg 2013-10-26, 03:51
+
Jason Rosenberg 2013-10-29, 20:29
+
Jason Rosenberg 2013-10-29, 20:39
+
Joel Koshy 2013-11-01, 20:43
+
Neha Narkhede 2013-11-01, 21:00
+
Jason Rosenberg 2013-11-02, 05:36
+
Jun Rao 2013-11-03, 03:41
+
Jason Rosenberg 2013-11-03, 11:24
+
Jun Rao 2013-11-04, 00:10
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB