Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Kafka 0.8 producer: Failed to send messages after 3 tries


+
Markus Roder 2013-06-17, 14:37
+
Jun Rao 2013-06-18, 04:29
Copy link to this message
-
Re: Kafka 0.8 producer: Failed to send messages after 3 tries
Ok, then it seems as I do not have the latest API, as if I'm using
"metadata.broker.list" I got the error " requirement failed: Missing
required property 'broker.list' "
Let me check if I can figure out on my side, what is going wrong.

Thanks for your hint.
regards
Markus
2013/6/18 Jun Rao <[EMAIL PROTECTED]>

> Are you using the latest 0.8 code? If so, broker.list is now changed to
> metadata.broker.list (see
> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Examplefor
> an example).
>
> Thanks,
>
> Jun
>
>
> On Mon, Jun 17, 2013 at 7:36 AM, Markus Roder <[EMAIL PROTECTED]
> >wrote:
>
> > Hi together,
> >
> > I'm collecting currently my first expirience with kafka   for a whole no
> > project in our company.
> >
> > We already had a first alpha version of our new system running with kafka
> > version 7.2. But as the kafka version 0.8 will support replication, we
> > thought about already testing this version (as we wanted to avoid setting
> > up drbd on version 7.2).
> >
> > so far, so good.
> > I started building the package as described in
> >
> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.8+Quick+Startand
> > was able to setup the 0.8 kafka-cluster without any problems.
> > Also the delivered console-producers and -consumer is working as
> expected.
> >
> > But now I also wrote my own producer - a very basic and simple one,
> without
> > any replication or anything else:
> >
> >
> > import java.util.Properties;
> >
> > import kafka.javaapi.producer.Producer;
> > import kafka.producer.KeyedMessage;
> > import kafka.producer.ProducerConfig;
> >
> > public class kafkaWriterTest {
> >
> > public static void main(String[] args) {
> > Properties props = new Properties();
> > props.put("broker.list", "mm-ws-maro.machcorp.lan:9092");
> > props.put("serializer.class", "kafka.serializer.StringEncoder");
> > // props.put("partitioner.class",
> > "com.mach.mme.alpha.ngb.pgreader.SimplePartitioner");
> > // props.put("request.required.acks", "0");
> >
> > ProducerConfig config = new ProducerConfig(props);
> >
> > Producer<String, String> producer =  new Producer<String,
> String>(config);
> >
> > System.out.println(config.props().toString());
> >                 KeyedMessage<String, String> data = new
> > KeyedMessage<String, String>( "m", "test1" );
> >                 producer.send(data);
> > producer.close();
> > }
> > }
> >
> > Unfortunately the above code always throws me the following runtime
> > exception:
> >
> > Exception in thread "main" kafka.common.FailedToSendMessageException:
> > Failed to send messages after 3 tries.
> > at
> >
> >
> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:88)
> > at kafka.producer.Producer.send(Producer.scala:74)
> > at kafka.javaapi.producer.Producer.send(Producer.scala:32)
> > at
> >
> >
> com.mach.mme.alpha.ngb.pgreader.kafkaWriterTest.main(kafkaWriterTest.java:24)
> >
> > But as I have a consumer already connected to the topic, I can see the
> one
> > sent message appearing four times.
> >
> > in the kafka-logs I can only see the following:
> > [2013-06-17 16:21:41,812] INFO Closing socket connection to /10.2.133.50
> .
> > (kafka.network.Processor)
> > [2013-06-17 16:21:41,978] INFO Closing socket connection to /10.2.133.50
> .
> > (kafka.network.Processor)
> > [2013-06-17 16:21:41,978] INFO Closing socket connection to /10.2.133.50
> .
> > (kafka.network.Processor)
> > [2013-06-17 16:21:42,083] INFO Closing socket connection to /10.2.133.50
> .
> > (kafka.network.Processor)
> > [2013-06-17 16:21:42,084] INFO Closing socket connection to /10.2.133.50
> .
> > (kafka.network.Processor)
> > [2013-06-17 16:21:42,189] INFO Closing socket connection to /10.2.133.50
> .
> > (kafka.network.Processor)
> >
> > This I could not find in the 7.2 version at all.
> > Can you assist me on my problem and maybe give me a hint, what could be
> the
> > problem on my setup?
> >
> > Notes:
> > - tried with a single instance cluster and with 3 kafka instances

 
+
Markus Roder 2013-06-18, 09:27
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB