Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Re: Kafka and Countandra


Copy link to this message
-
Re: Kafka and Countandra
Thanks
https://issues.apache.org/jira/browse/KAFKA-279

Regards
Milind
On Sun, Feb 19, 2012 at 10:55 AM, Jun Rao <[EMAIL PROTECTED]> wrote:

> Milind,
>
> You hit a couple of bugs in ConsoleProducer.
>
> 1. The code does the following to set batch size
>      props.put("batch.size", batchSize)
> Instead, it should do
>      props.put("batch.size", batchSize.toString)
>
> 2. It sets the wrong property name for timeout. Instead of doing
>    props.put("queue.enqueueTimeout.ms", sendTimeout.toString)
> it should do
>    props.put("queue.time", sendTimeout.toString)
>
> Could you file a jira for that?
>
> Thanks,
>
> Jun
>
>
> On Sat, Feb 18, 2012 at 11:35 PM, Milind Parikh <[EMAIL PROTECTED]
> >wrote:
>
> > I am experimenting with Kafka to see if it can serve as a event buffer
> for
> > Countandra (www.countandra.org) and beyond that, into cascading
> counters.
> > I
> > have setup Kafka on a local server Ubuntu 10.04, openjdk1.6 with default
> > installation of 0.7.
> >
> > *1. While the default console-producer, console-consumer paradigm works
> > great, when I try modiying the batch size
> > *
> > bin/kafka-console-producer.sh --batch-size 300   --zookeeper
> localhost:2181
> > --topic test1
> >
> > it gives me a
> >
> > Exception in thread "main" java.lang.NumberFormatException: null
> >    at java.lang.Integer.parseInt(Integer.java:443)
> >    at java.lang.Integer.parseInt(Integer.java:514)
> >    at
> > scala.collection.immutable.StringLike$class.toInt(StringLike.scala:207)
> >    at scala.collection.immutable.StringOps.toInt(StringOps.scala:31)
> >    at kafka.utils.Utils$.getIntInRange(Utils.scala:189)
> >    at kafka.utils.Utils$.getInt(Utils.scala:174)
> >    at
> >
> >
> kafka.producer.async.AsyncProducerConfigShared$class.$init$(AsyncProducerConfig.scala:45)
> >    at kafka.producer.ProducerConfig.<init>(ProducerConfig.scala:25)
> >    at kafka.producer.ConsoleProducer$.main(ConsoleProducer.scala:108)
> >    at kafka.producer.ConsoleProducer.main(ConsoleProducer.scala)
> >
> > I have looked at the code and can't figure out what's wrong
> >
> > *2. When I do bin/kafka-console-producer.sh --timeout 30000   --zookeeper
> > localhost:2181 --topic test1 *
> >
> > I would think that console-producer would wait for 30s if the batch size
> > (default 200) is not full. It doesn't. It takes the same time without the
> > timeout parameter (default 1000) and dumps whatever the batch size.
> >
> > I am probably missing something here. Any pointers would be appreciated.
> >
> > Thanks
> > Milind
> >
>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB