-Re: Compression in kafka 0.8
Jason Rosenberg 2013-06-05, 07:14
In this case, does the consumer code need to change, account for the
compression (or is this handled automatically in the consumer apis?).
If we decide to start compressing messages to a topic, can a consumer
seamlessly move between the transition from uncompressed to compressed?
On Tue, Jun 4, 2013 at 11:16 PM, Vadim Keylis <[EMAIL PROTECTED]> wrote:
> Thanks so much Neha
> On Tue, Jun 4, 2013 at 9:57 PM, Neha Narkhede <[EMAIL PROTECTED]
> > The only way of storing compressed data on the server is to have the
> > producer send it compressed. The server sends the data in compressed
> > to the consumers.
> > Thanks,
> > Neha
> > On Tue, Jun 4, 2013 at 9:55 PM, Vadim Keylis <[EMAIL PROTECTED]>
> > wrote:
> > > Good evening. I would like to preserve logs in kafka for 2 days. Due to
> > the
> > > size of data it must be compressed otherwise it will take enormous
> > > of disk space. Its my understanding that the only way to store messages
> > in
> > > kafka compressed is to enable compression on the producer side, is that
> > > correct? Will they be sent to consumer also in compressed format? How
> > > consumer can check if stream been sent compressed or not? Are there
> > > options that will allow storing data in kafka for n number of days
> > > compressed?
> > >
> > > Thanks so much,
> > > Vadim
> > >