Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Kafka 0.8 producer: Failed to send messages after 3 tries


Copy link to this message
-
Re: Kafka 0.8 producer: Failed to send messages after 3 tries
Markus Roder 2013-06-18, 09:27
Hi Jun,

thanks for your hint, with the newest 0.8beta1 code it is now working.
regards
Markus
2013/6/18 Markus Roder <[EMAIL PROTECTED]>

> Ok, then it seems as I do not have the latest API, as if I'm using
> "metadata.broker.list" I got the error " requirement failed: Missing
> required property 'broker.list' "
> Let me check if I can figure out on my side, what is going wrong.
>
> Thanks for your hint.
> regards
> Markus
>
>
> 2013/6/18 Jun Rao <[EMAIL PROTECTED]>
>
>> Are you using the latest 0.8 code? If so, broker.list is now changed to
>> metadata.broker.list (see
>>
>> https://cwiki.apache.org/confluence/display/KAFKA/0.8.0+Producer+Examplefor
>> an example).
>>
>> Thanks,
>>
>> Jun
>>
>>
>> On Mon, Jun 17, 2013 at 7:36 AM, Markus Roder <[EMAIL PROTECTED]
>> >wrote:
>>
>> > Hi together,
>> >
>> > I'm collecting currently my first expirience with kafka   for a whole no
>> > project in our company.
>> >
>> > We already had a first alpha version of our new system running with
>> kafka
>> > version 7.2. But as the kafka version 0.8 will support replication, we
>> > thought about already testing this version (as we wanted to avoid
>> setting
>> > up drbd on version 7.2).
>> >
>> > so far, so good.
>> > I started building the package as described in
>> >
>> https://cwiki.apache.org/confluence/display/KAFKA/Kafka+0.8+Quick+Startand
>> > was able to setup the 0.8 kafka-cluster without any problems.
>> > Also the delivered console-producers and -consumer is working as
>> expected.
>> >
>> > But now I also wrote my own producer - a very basic and simple one,
>> without
>> > any replication or anything else:
>> >
>> >
>> > import java.util.Properties;
>> >
>> > import kafka.javaapi.producer.Producer;
>> > import kafka.producer.KeyedMessage;
>> > import kafka.producer.ProducerConfig;
>> >
>> > public class kafkaWriterTest {
>> >
>> > public static void main(String[] args) {
>> > Properties props = new Properties();
>> > props.put("broker.list", "mm-ws-maro.machcorp.lan:9092");
>> > props.put("serializer.class", "kafka.serializer.StringEncoder");
>> > // props.put("partitioner.class",
>> > "com.mach.mme.alpha.ngb.pgreader.SimplePartitioner");
>> > // props.put("request.required.acks", "0");
>> >
>> > ProducerConfig config = new ProducerConfig(props);
>> >
>> > Producer<String, String> producer =  new Producer<String,
>> String>(config);
>> >
>> > System.out.println(config.props().toString());
>> >                 KeyedMessage<String, String> data = new
>> > KeyedMessage<String, String>( "m", "test1" );
>> >                 producer.send(data);
>> > producer.close();
>> > }
>> > }
>> >
>> > Unfortunately the above code always throws me the following runtime
>> > exception:
>> >
>> > Exception in thread "main" kafka.common.FailedToSendMessageException:
>> > Failed to send messages after 3 tries.
>> > at
>> >
>> >
>> kafka.producer.async.DefaultEventHandler.handle(DefaultEventHandler.scala:88)
>> > at kafka.producer.Producer.send(Producer.scala:74)
>> > at kafka.javaapi.producer.Producer.send(Producer.scala:32)
>> > at
>> >
>> >
>> com.mach.mme.alpha.ngb.pgreader.kafkaWriterTest.main(kafkaWriterTest.java:24)
>> >
>> > But as I have a consumer already connected to the topic, I can see the
>> one
>> > sent message appearing four times.
>> >
>> > in the kafka-logs I can only see the following:
>> > [2013-06-17 16:21:41,812] INFO Closing socket connection to /
>> 10.2.133.50.
>> > (kafka.network.Processor)
>> > [2013-06-17 16:21:41,978] INFO Closing socket connection to /
>> 10.2.133.50.
>> > (kafka.network.Processor)
>> > [2013-06-17 16:21:41,978] INFO Closing socket connection to /
>> 10.2.133.50.
>> > (kafka.network.Processor)
>> > [2013-06-17 16:21:42,083] INFO Closing socket connection to /
>> 10.2.133.50.
>> > (kafka.network.Processor)
>> > [2013-06-17 16:21:42,084] INFO Closing socket connection to /
>> 10.2.133.50.
>> > (kafka.network.Processor)
>> > [2013-06-17 16:21:42,189] INFO Closing socket connection to /
>> 10.2.133.50.