Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Topic creation on restart


Copy link to this message
-
Re: Topic creation on restart
Ok, so this can happen, even if the node has not been placed back into
rotation, at the metadata vip?
On Tue, Nov 5, 2013 at 12:11 AM, Neha Narkhede <[EMAIL PROTECTED]>wrote:

> It is probably due to the same metadata propagation issue.
> https://issues.apache.org/jira/browse/KAFKA-1111 should fix that.
>
> Thanks,
> Neha
>
>
> On Mon, Nov 4, 2013 at 11:47 AM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>
> > Ok,
> >
> > After adding a delay before enabling a freshly started broker in the
> > metadata vip that clients use, it seems to have drastically reduced the
> > number of these topic creation requests.  However, not all of them.
> >
> > I still see (only some times) a handful of "Topic creation" log messages,
> > that happen on a freshly started broker, during the time kafka has
> started,
> > but before the broker has been enabled in the metadata VIP.  Is there an
> > explanation for this?
> >
> > Thanks!
> >
> > Jason
> >
> >
> > On Fri, Nov 1, 2013 at 4:45 PM, Neha Narkhede <[EMAIL PROTECTED]
> > >wrote:
> >
> > > The mbeans are explained here -
> > > http://kafka.apache.org/documentation.html#monitoring. Look for
> > > *QueueTimeMs
> > >
> > > Thanks,
> > > Neha
> > >
> > >
> > > On Fri, Nov 1, 2013 at 12:14 PM, Jason Rosenberg <[EMAIL PROTECTED]>
> > wrote:
> > >
> > > > Neha,
> > > >
> > > > This cluster has on the order of 750 topics.
> > > >
> > > > It looks like if I add a 20 second delay before placing a broker into
> > the
> > > > vip for metadata requests, it never seems to have this issue.  So I'm
> > not
> > > > sure about the 104 seconds number, other than that was how long the
> > flood
> > > > of "Topic creation" log messages went on for (over 500 of these).
> > > >
> > > > Which metric should I look at for 'high queue time'?
> > > >
> > > > Jason
> > > >
> > > >
> > > > On Fri, Nov 1, 2013 at 5:54 AM, Neha Narkhede <
> [EMAIL PROTECTED]
> > > > >wrote:
> > > >
> > > > > It is proportional to the number of topics but still seems too
> long.
> > > Did
> > > > > the broker have high queue time on all requests? Also how many
> topics
> > > > > existed on this cluster?
> > > > >
> > > > > Thanks,
> > > > > Neha
> > > > > On Oct 31, 2013 10:56 PM, "Jason Rosenberg" <[EMAIL PROTECTED]>
> > wrote:
> > > > >
> > > > > > In this case, it appears to have gone on for 104 seconds.  Should
> > it
> > > > take
> > > > > > that long?  It doesn't seem to always take this long.  I guess
> one
> > of
> > > > the
> > > > > > drawbacks of having a large number of topics (brings out the edge
> > > > cases).
> > > > > >
> > > > > > Filed: https://issues.apache.org/jira/browse/KAFKA-1111
> > > > > >
> > > > > >
> > > > > > On Fri, Nov 1, 2013 at 1:17 AM, Neha Narkhede <
> > > [EMAIL PROTECTED]
> > > > > > >wrote:
> > > > > >
> > > > > > > >> Would it be possible for the broker to make sure all
> metadata
> > is
> > > > > > > propagated, before completing the startup sequence, and taking
> on
> > > > > > metadata
> > > > > > > requests?
> > > > > > >
> > > > > > > Yes, that is the flaw I mentioned before. We need to fix it.
> > > > > > >
> > > > > > >
> > > > > > > On Thu, Oct 31, 2013 at 9:56 PM, Jason Rosenberg <
> > [EMAIL PROTECTED]
> > > >
> > > > > > wrote:
> > > > > > >
> > > > > > > > Ok,
> > > > > > > >
> > > > > > > > So, I can safely ignore these, it sounds like.  I don't see
> any
> > > > > > > > corresponding logging around it subsequently not succeeding
> to
> > > > > actually
> > > > > > > > create the topic in zk.
> > > > > > > >
> > > > > > > > Would it be possible for the broker to make sure all metadata
> > is
> > > > > > > > propagated, before completing the startup sequence, and
> taking
> > on
> > > > > > > metadata
> > > > > > > > requests?
> > > > > > > >
> > > > > > > > If I were to not place the newly started broker back into the
> > vip
> > > > > for a
> > > > > > > > delay period after the kafka broker has started, would that
> > solve
> > > > > this
> > > > > > > > issue?
> > > > > > > >