Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Kafka and Countandra


Copy link to this message
-
Re: Kafka and Countandra
Thanks
https://issues.apache.org/jira/browse/KAFKA-279

Regards
Milind
On Sun, Feb 19, 2012 at 10:55 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Milind,
>
> You hit a couple of bugs in ConsoleProducer.
>
> 1. The code does the following to set batch size
>      props.put("batch.size", batchSize)
> Instead, it should do
>      props.put("batch.size", batchSize.toString)
>
> 2. It sets the wrong property name for timeout. Instead of doing
>    props.put("queue.enqueueTimeout.ms", sendTimeout.toString)
> it should do
>    props.put("queue.time", sendTimeout.toString)
>
> Could you file a jira for that?
>
> Thanks,
>
> Jun
>
>
> On Sat, Feb 18, 2012 at 11:35 PM, Milind Parikh <[EMAIL PROTECTED]
> >wrote:
>
> > I am experimenting with Kafka to see if it can serve as a event buffer
> for
> > Countandra (www.countandra.org) and beyond that, into cascading
> counters.
> > I
> > have setup Kafka on a local server Ubuntu 10.04, openjdk1.6 with default
> > installation of 0.7.
> >
> > *1. While the default console-producer, console-consumer paradigm works
> > great, when I try modiying the batch size
> > *
> > bin/kafka-console-producer.sh --batch-size 300   --zookeeper
> localhost:2181
> > --topic test1
> >
> > it gives me a
> >
> > Exception in thread "main" java.lang.NumberFormatException: null
> >    at java.lang.Integer.parseInt(Integer.java:443)
> >    at java.lang.Integer.parseInt(Integer.java:514)
> >    at
> > scala.collection.immutable.StringLike$class.toInt(StringLike.scala:207)
> >    at scala.collection.immutable.StringOps.toInt(StringOps.scala:31)
> >    at kafka.utils.Utils$.getIntInRange(Utils.scala:189)
> >    at kafka.utils.Utils$.getInt(Utils.scala:174)
> >    at
> >
> >
> kafka.producer.async.AsyncProducerConfigShared$class.$init$(AsyncProducerConfig.scala:45)
> >    at kafka.producer.ProducerConfig.<init>(ProducerConfig.scala:25)
> >    at kafka.producer.ConsoleProducer$.main(ConsoleProducer.scala:108)
> >    at kafka.producer.ConsoleProducer.main(ConsoleProducer.scala)
> >
> > I have looked at the code and can't figure out what's wrong
> >
> > *2. When I do bin/kafka-console-producer.sh --timeout 30000   --zookeeper
> > localhost:2181 --topic test1 *
> >
> > I would think that console-producer would wait for 30s if the batch size
> > (default 200) is not full. It doesn't. It takes the same time without the
> > timeout parameter (default 1000) and dumps whatever the batch size.
> >
> > I am probably missing something here. Any pointers would be appreciated.
> >
> > Thanks
> > Milind
> >
>