Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Kafka 0.8 producer: Failed to send messages after 3 tries


Copy link to this message
-
Kafka 0.8 producer: Failed to send messages after 3 tries
Hi together,

I'm collecting currently my first expirience with kafka   for a whole no
project in our company.

We already had a first alpha version of our new system running with kafka
version 7.2. But as the kafka version 0.8 will support replication, we
thought about already testing this version (as we wanted to avoid setting
up drbd on version 7.2).

so far, so good.
I started building the package as described in
https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.8+Quick+Start and
was able to setup the 0.8 kafka-cluster without any problems.
Also the delivered console-producers and -consumer is working as expected.

But now I also wrote my own producer - a very basic and simple one, without
any replication or anything else:
import java.util.Properties;

import kafka.javaapi.producer.Producer;
import kafka.producer.KeyedMessage;
import kafka.producer.ProducerConfig;

public class kafkaWriterTest {

public static void main(String[] args) {
Properties props = new Properties();
props.put("broker.list", "mm-ws-maro.machcorp.lan:9092");
props.put("serializer.class", "kafka.serializer.StringEncoder");
// props.put("partitioner.class",
"com.mach.mme.alpha.ngb.pgreader.SimplePartitioner");
// props.put("request.required.acks", "0");

ProducerConfig config = new ProducerConfig(props);

Producer<String, String> producer =  new Producer<String, String>(config);

System.out.println(config.props().toString());
                KeyedMessage<String, String> data = new
KeyedMessage<String, String>( "m", "test1" );
                producer.send(data);
producer.close();
}
}

Unfortunately the above code always throws me the following runtime
exception:

Exception in thread "main" kafka.common.FailedToSendMessageException:
Failed to send messages after 3 tries.
at
kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:88)
at kafka.producer.Producer.send(Producer.scala:74)
at kafka.javaapi.producer.Producer.send(Producer.scala:32)
at
com.mach.mme.alpha.ngb.pgreader.kafkaWriterTest.main(kafkaWriterTest.java:24)

But as I have a consumer already connected to the topic, I can see the one
sent message appearing four times.

in the kafka-logs I can only see the following:
[2013-06-17 16:21:41,812] INFO Closing socket connection to /10.2.133.50.
(kafka.network.Processor)
[2013-06-17 16:21:41,978] INFO Closing socket connection to /10.2.133.50.
(kafka.network.Processor)
[2013-06-17 16:21:41,978] INFO Closing socket connection to /10.2.133.50.
(kafka.network.Processor)
[2013-06-17 16:21:42,083] INFO Closing socket connection to /10.2.133.50.
(kafka.network.Processor)
[2013-06-17 16:21:42,084] INFO Closing socket connection to /10.2.133.50.
(kafka.network.Processor)
[2013-06-17 16:21:42,189] INFO Closing socket connection to /10.2.133.50.
(kafka.network.Processor)

This I could not find in the 7.2 version at all.
Can you assist me on my problem and maybe give me a hint, what could be the
problem on my setup?

Notes:
- tried with a single instance cluster and with 3 kafka instances
- also tried to change some properties in the producer, but could not
identify yet, what the problem exactly is.
- maybe interesting: producer and kafka-cluster running on same node.

If you need further information, I'll be happy to provide you with all the
information you need.

btw. @ developers: thanks for your good job.

regards
Markus

--
Markus Roder
Mail: [EMAIL PROTECTED]
Profil: http://gplus.to/markusroder

 
+
Jun Rao 2013-06-18, 04:29
+
Markus Roder 2013-06-18, 07:27
+
Markus Roder 2013-06-18, 09:27