Kafka, mail # user - Re: Sending huge binary files via Kafka? - 2013-09-04, 14:26
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
Re: Sending huge binary files via Kafka?
The message size limit is imposed to protect the brokers and consumers from
running out of memory. The consumer does not have support for streaming a
message and has to allocate memory to be able to read the largest message.
You could try compressing the files but I'm not sure if that will get you
as much space saving to make it feasible for Kafka usage.

On Sep 4, 2013 5:29 AM, "Maier, Dr. Andreas" <[EMAIL PROTECTED]>
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB