Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Getting LeaderNotAvailableException in console producer after increasing partitions from 4 to 16.


Copy link to this message
-
Re: Getting LeaderNotAvailableException in console producer after increasing partitions from 4 to 16.
Thanks, This is small fix to ConsoleProducer.scala only. Will use 0.8
branch.

Thanks,
Raja.
On Wed, Aug 28, 2013 at 12:49 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:

> Rajasekar,
>
> We are trying to minimize the number of patches in 0.8 to critical bug
> fixes or broken tooling. If the patch involves significant code changes, we
> would encourage taking it on trunk. If you want to just fix the console
> producer to take the retry argument, I would think it is small enough to
> consider taking it on 0.8 branch since it affects the usability of the
> console producer.
>
> Thanks,
> Neha
>
>
> On Wed, Aug 28, 2013 at 8:36 AM, Rajasekar Elango <[EMAIL PROTECTED]
> >wrote:
>
> > Guozhang ,
> >
> > *The documentation says I need to work off of trunk. Can you confirm If I
> > should be working in trunk or different branch.*
> > *
> > *
> > *Thanks,*
> > *Raja.*
> >
> >
> > On Tue, Aug 27, 2013 at 8:33 PM, Guozhang Wang <[EMAIL PROTECTED]>
> wrote:
> >
> > > Cool! You can follow the process of creating a JIRA here:
> > >
> > > http://kafka.apache.org/contributing.html
> > >
> > > And submit patch here:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/Git+Workflow
> > >
> > > It will be great if you can also add an entry for this issue in FAQ
> > since I
> > > think this is a common question:
> > >
> > > https://cwiki.apache.org/confluence/display/KAFKA/FAQ
> > >
> > > Guozhang
> > >
> > >
> > > On Tue, Aug 27, 2013 at 2:38 PM, Rajasekar Elango <
> > [EMAIL PROTECTED]
> > > >wrote:
> > >
> > > > Thanks Guozhang, Changing max retry to 5 worked. Since I am changing
> > > > console producer code, I can also submit patch adding both
> > > > message.send.max.retries
> > > > and retry.backoff.ms to console producer. Can you let me know
> process
> > > for
> > > > submitting patch?
> > > >
> > > > Thanks,
> > > > Raja.
> > > >
> > > >
> > > > On Tue, Aug 27, 2013 at 4:03 PM, Guozhang Wang <[EMAIL PROTECTED]>
> > > wrote:
> > > >
> > > > > Hello Rajasekar,
> > > > >
> > > > > The remove fetcher log entry is normal under addition of
> partitions,
> > > > since
> > > > > they indicate that some leader changes have happened so brokers are
> > > > closing
> > > > > the fetchers to the old leaders.
> > > > >
> > > > > I just realized that the console Producer does not have the
> > > > > message.send.max.retries options yet. Could you file a JIRA for
> this
> > > and
> > > > I
> > > > > will followup to add this option? As for now you can hard modify
> the
> > > > > default value from 3 to a larger number.
> > > > >
> > > > > Guozhang
> > > > >
> > > > >
> > > > > On Tue, Aug 27, 2013 at 12:37 PM, Rajasekar Elango
> > > > > <[EMAIL PROTECTED]>wrote:
> > > > >
> > > > > > Thanks Neha & Guozhang,
> > > > > >
> > > > > > When I ran StateChangeLogMerger, I am seeing this message
> repeated
> > 16
> > > > > times
> > > > > > for each partition:
> > > > > >
> > > > > > [2013-08-27 12:30:02,535] INFO [ReplicaFetcherManager on broker
> 1]
> > > > > Removing
> > > > > > fetcher for partition [test-60,13]
> > > (kafka.server.ReplicaFetcherManager)
> > > > > > [2013-08-27 12:30:02,536] INFO [Log Manager on Broker 1] Created
> > log
> > > > for
> > > > > > partition [test-60,13] in
> > > > > >
> > > /home/relango/dev/mandm/kafka/main/target/dist/mandm-kafka/kafka-data.
> > > > > > (kafka.log.LogManager)
> > > > > >
> > > > > > I am also seeing .log and .index files created for this topic in
> > data
> > > > > dir.
> > > > > > Also list topic command shows leaders, replicas and isrs for all
> > > > > > partitions. Do you still think increasing num of retries would
> help
> > > or
> > > > is
> > > > > > it some other issue..? Also console Producer doesn't seem to
>  have
> > > > option
> > > > > > to set num of retries. Is there a way to configure num of retries
> > for
> > > > > > console producer ?
> > > > > >
> > > > > > Thanks,
> > > > > > Raja.
> > > > > >
> > > > > >
> > > > > > On Tue, Aug 27, 2013 at 12:52 PM, Neha Narkhede <

Thanks,
Raja.