Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Producer.send questions


Copy link to this message
-
Re: Producer.send questions
Jun,

There are several others I've seen that I would have thought would be
retryable (possibly after an exponential backoff delay).  I'm curious about:

BrokerNotAvailableException
FailedToSendMessageException
QueueFullException (happens if producerType is 'async')
KafkaException (this seems to wrap lots of base conditions, does one have
to sort through the different wrapped exception types?)
LeaderNotAvailableException
MessageSizeTooLargeException (does a batch of messages get treated as a
single message, when checking for message size too large?)
ReplicaNotAvailableException
UnavailableProducerException
UnknownException

Also, what about my first question, regarding whether it makes sense to
refresh a producer by closing it and restarting it after a failure?

Thanks,

Jason
On Fri, Aug 23, 2013 at 9:07 PM, Jun Rao <[EMAIL PROTECTED]> wrote:

> For the most part, only SocketExceptions and NotLeaderForPartitionException
> are recoverable. MessageSizeTooLargeException may be recoverable with a
> smaller batch size.
>
> Thanks,
>
> Jun
>
>
> On Fri, Aug 23, 2013 at 4:09 PM, Jason Rosenberg <[EMAIL PROTECTED]> wrote:
>
> > I'm using the kafka.javaapi.producer.Producer class from a java client.
> >  I'm wondering if it ever makes sense to refresh a producer by stopping
> it
> > and creating a new one, for example in response to a downstream IO error
> > (e.g. a broker got restarted, or a stale socket, etc.).
> >
> > Or should it always be safe to rely on the producer's implementation to
> > manage it's pool of BlockingChannel connections, etc.
> >
> > I'm also interested in trying to understand which exceptions indicate a
> > failed send() request might be retryable (basically anything that doesn't
> > involve a data-dependent problem, like a malformed message, or a message
> > too large, etc.).
> >
> > Unfortunately, the range of Exceptions that can be thrown by the various
> > javaapi methods is not yet well documented.  It would be nice to have
> some
> > notion of whether an exception is the result of a data error, or a
> > transient downstream connection error, etc.
> >
> > Jason
> >
>