Kafka, mail # user - Re: Sending huge binary files via Kafka? - 2013-09-04, 14:26
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
Re: Sending huge binary files via Kafka?
The message size limit is imposed to protect the brokers and consumers from
running out of memory. The consumer does not have support for streaming a
message and has to allocate memory to be able to read the largest message.
You could try compressing the files but I'm not sure if that will get you
as much space saving to make it feasible for Kafka usage.

Thanks,
Neha
On Sep 4, 2013 5:29 AM, "Maier, Dr. Andreas" <[EMAIL PROTECTED]>
wrote:
 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB