Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka, mail # user - Kafka compression issues


Copy link to this message
-
Re: Kafka compression issues
Jun Rao 2013-07-10, 04:14
Any error in the producer and the broker log (including state-change.log)?

Thanks,

Jun
On Tue, Jul 9, 2013 at 12:31 PM, arathi maddula <[EMAIL PROTECTED]>wrote:

> Hi,
>
>
>
> I use kafka 0.8. When I run the kafka console producer using
>
>
>
>  ./kafka-console-producer.sh --topic test.compress.e --compress true
>  --broker-list 127.0.0.1:9092
>
>
>
> Iam able to see compressed messages in the log.
>
>
> But when I run a Java producer class using the following properties, no
> message is getting into the data log for that topic.
> Please tell me what do I need to get the compressed message into log using
> java producer
> This the code snippet:
>
>
>
>
>
> Properties props = *new* Properties();
>
>
>
> props.put("broker.list", "127.0.0.1:9092");
>
>
>
>
>
> props.put("serializer.class", "kafka.serializer.StringEncoder");
>
> props.put("compress", "true");
>
> props.put("compression.codec", "gzip");
>
> props.put("compressed.topics", "test.compress22");
>
>
>
> ProducerConfig config = *new* ProducerConfig(props);
>
> Producer<String, String> producer = *new* Producer<String, String>(config);
>
> String KafkaTopic = "test.compress22";
>
> String strLine;
>
> *try* {
>
> *for*(*int* i=1;i<=2; i++){
>
> strLine="Message "+I;
>
> KeyedMessage<String, String> s = *new* KeyedMessage<String,
> String>(KafkaTopic, *null*, strLine);
>
> producer.send(s);
>
>
>
>
>
>
>
> }
>
> producer.close();
>
>
>
> } *catch* (Exception e) {
>
> e.printStackTrace();
>
> }
>
>
>  Thanks
> Arathi
>