Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # dev >> Kafka and Countandra


Copy link to this message
-
Kafka and Countandra
I am experimenting with Kafka to see if it can serve as a event buffer for
Countandra (www.countandra.org) and beyond that, into cascading counters. I
have setup Kafka on a local server Ubuntu 10.04, openjdk1.6 with default
installation of 0.7.

*1. While the default console-producer, console-consumer paradigm works
great, when I try modiying the batch size
*
bin/kafka-console-producer.sh --batch-size 300   --zookeeper localhost:2181
--topic test1

it gives me a

Exception in thread "main" java.lang.NumberFormatException: null
    at java.lang.Integer.parseInt(Integer.java:443)
    at java.lang.Integer.parseInt(Integer.java:514)
    at
scala.collection.immutable.StringLike$class.toInt(StringLike.scala:207)
    at scala.collection.immutable.StringOps.toInt(StringOps.scala:31)
    at kafka.utils.Utils$.getIntInRange(Utils.scala:189)
    at kafka.utils.Utils$.getInt(Utils.scala:174)
    at
kafka.producer.async.AsyncProducerConfigShared$class.$init$(AsyncProducerConfig.scala:45)
    at kafka.producer.ProducerConfig.<init>(ProducerConfig.scala:25)
    at kafka.producer.ConsoleProducer$.main(ConsoleProducer.scala:108)
    at kafka.producer.ConsoleProducer.main(ConsoleProducer.scala)

I have looked at the code and can't figure out what's wrong

*2. When I do bin/kafka-console-producer.sh --timeout 30000   --zookeeper
localhost:2181 --topic test1 *

I would think that console-producer would wait for 30s if the batch size
(default 200) is not full. It doesn't. It takes the same time without the
timeout parameter (default 1000) and dumps whatever the batch size.

I am probably missing something here. Any pointers would be appreciated.

Thanks
Milind