Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> 0.8.0 producer can't connect to cluster?


+
Chris Curtin 2012-11-27, 15:17
+
Jun Rao 2012-11-27, 16:49
+
Chris Curtin 2012-11-27, 16:56
Copy link to this message
-
Re: 0.8.0 producer can't connect to cluster?
When the producer fails to send in 3 retries, it will log an error. Before
that, in the producer, we log the failure of each of the retries in warn
and it tells you the cause.

Also, did you create the topic first?

Thanks,

Jun

On Tue, Nov 27, 2012 at 8:56 AM, Chris Curtin <[EMAIL PROTECTED]>wrote:

> The error from when it fails is all I get. Nothing on the broker side and
> no other errors or exceptions on the client.
>
> Where should I be looking for the reasons? Is there a callback I should be
> implementing?
>
> Thanks,
>
> Chris
>
>
> On Tue, Nov 27, 2012 at 11:49 AM, Jun Rao <[EMAIL PROTECTED]> wrote:
>
> > Do you have the exception that caused each retry to fail?
> >
> > Thanks,
> >
> > Jun
> >
> > On Tue, Nov 27, 2012 at 7:17 AM, Chris Curtin <[EMAIL PROTECTED]
> > >wrote:
> >
> > > Hi,
> > >
> > > Installed 0.8.0 yesterday, 3 physical machines, 9 brokers running (3
> per
> > > machine). 2 topics, with 3 replicas each
> > >
> > > the console producer/consumer examples work fine.
> > >
> > > When I run my producer logic I get the following error whether on the
> > > cluster or in my dev environment:
> > >
> > > Exception in thread "main" kafka.common.FailedToSendMessageException:
> > > Failed to send messages after 3 tries.
> > > at
> > >
> > >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:70)
> > > at kafka.producer.Producer.send(Producer.scala:75)
> > > at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> > > at
> > com.silverpop.kafka.playproducer.TestProducer.main(TestProducer.java:40)
> > > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> > > at
> > >
> > >
> >
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
> > > at
> > >
> > >
> >
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
> > > at java.lang.reflect.Method.invoke(Method.java:597)
> > > at com.intellij.rt.execution.application.AppMain.main(AppMain.java:120)
> > >
> > > Code is pretty basic:
> > >
> > > public class TestProducer {
> > >     public static void main(String[] args) {
> > >
> > >         String zookeeper = args[0];
> > >         long events = Long.parseLong(args[1]);
> > >         long blocks = Long.parseLong(args[2]);
> > >
> > >         Random rnd = new Random();
> > >
> > >         Properties props = new Properties();
> > >         props.put("broker.list", "mongodb03.atlnp1:9092");
> > >
> > >
> > >         props.put("serializer.class",
> "kafka.serializer.StringEncoder");
> > >         ProducerConfig config = new ProducerConfig(props);
> > >         Producer<Integer, String> producer = new Producer<Integer,
> > > String>(config);
> > >
> > >
> > >         for (long nBlocks = 0; nBlocks < blocks; nBlocks++) {
> > >             for (long nEvents = 0; nEvents < events; nEvents++) {
> > >                 long runtime = new Date().getTime();
> > >                 String msg = runtime + "," + (50 + nBlocks) + "," +
> > > nEvents+ "," + rnd.nextInt(1000);
> > >                 KeyedMessage<Integer, String> data = new
> > > KeyedMessage<Integer, String>("test1", msg);
> > >                 producer.send(data);
> > >             }
> > >         }
> > >
> > >
> > >     }
> > >
> > > using Zookeeper doesn't matter. Changing broker.list to include all 9
> > > brokers doesn't matter. Changing Producer and KeyedMessage to be
> <String,
> > > String> doesn't matter.
> > >
> > > Thoughts on what I'm doing wrong?
> > >
> > > Thanks,
> > >
> > > Chris
> > >
> >
>
+
Chris Curtin 2012-11-27, 18:12
+
Jay Kreps 2012-11-27, 18:17
+
Jason Rosenberg 2012-11-27, 18:20
+
Jay Kreps 2012-11-27, 18:30
+
Chris Curtin 2012-11-27, 18:25
+
Chris Curtin 2012-11-27, 18:39
+
Chris Curtin 2012-11-27, 20:39
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB