Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Kafka compression issues


Copy link to this message
-
Re: Kafka compression issues
Any error in the producer and the broker log (including state-change.log)?

Thanks,

Jun
On Tue, Jul 9, 2013 at 12:31 PM, arathi maddula <[EMAIL PROTECTED]>wrote:

> Hi,
>
>
>
> I use kafka 0.8. When I run the kafka console producer using
>
>
>
>  ./kafka-console-producer.sh --topic test.compress.e --compress true
>  --broker-list 127.0.0.1:9092
>
>
>
> Iam able to see compressed messages in the log.
>
>
> But when I run a Java producer class using the following properties, no
> message is getting into the data log for that topic.
> Please tell me what do I need to get the compressed message into log using
> java producer
> This the code snippet:
>
>
>
>
>
> Properties props = *new* Properties();
>
>
>
> props.put("broker.list", "127.0.0.1:9092");
>
>
>
>
>
> props.put("serializer.class", "kafka.serializer.StringEncoder");
>
> props.put("compress", "true");
>
> props.put("compression.codec", "gzip");
>
> props.put("compressed.topics", "test.compress22");
>
>
>
> ProducerConfig config = *new* ProducerConfig(props);
>
> Producer<String, String> producer = *new* Producer<String, String>(config);
>
> String KafkaTopic = "test.compress22";
>
> String strLine;
>
> *try* {
>
> *for*(*int* i=1;i<=2; i++){
>
> strLine="Message "+I;
>
> KeyedMessage<String, String> s = *new* KeyedMessage<String,
> String>(KafkaTopic, *null*, strLine);
>
> producer.send(s);
>
>
>
>
>
>
>
> }
>
> producer.close();
>
>
>
> } *catch* (Exception e) {
>
> e.printStackTrace();
>
> }
>
>
>  Thanks
> Arathi
>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB