Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Compression in kafka 0.8

Copy link to this message
Re: Compression in kafka 0.8
The only way of storing compressed data on the server is to have the
producer send it compressed. The server sends the data in compressed format
to the consumers.

On Tue, Jun 4, 2013 at 9:55 PM, Vadim Keylis <[EMAIL PROTECTED]> wrote:

> Good evening. I would like to preserve logs in kafka for 2 days. Due to the
> size of data it must be compressed otherwise it will take enormous amount
> of disk space. Its my understanding that the only way to store messages in
> kafka compressed is to enable compression on the producer side, is that
> correct? Will they be sent to consumer also in compressed format? How the
> consumer can check if stream been sent compressed or not? Are there other
> options that will allow storing data in kafka for n number of days
> compressed?
> Thanks so much,
> Vadim