Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> Question on message content, compression, multiple messages per kafka message?


Copy link to this message
-
Re: Question on message content, compression, multiple messages per kafka message?
Using a single Kafka message to contain an application snapshot has the
upside of getting atomicity for free. Either the snapshot will be written
as a whole to Kafka or not. This is poor man's transactionality. Care needs
to be taken to ensure that the message is not too large since that might
cause memory consumption problems on the server or the consumers.

As far as compression overhead is concerned, have you tried running Snappy?
Snappy's performance is good enough to offset the decompression-compression
overhead on the server.

Thanks,
Neha
On Thu, Jun 26, 2014 at 12:42 PM, Bert Corderman <[EMAIL PROTECTED]> wrote: