Home | About | Sematext search-lucene.com search-hadoop.com search-devops.com
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> Kafka throw InvalidMessageException and lost data


+
Helin Xiang 2013-03-19, 03:22
+
Jun Rao 2013-03-19, 04:58
+
Helin Xiang 2013-03-19, 05:13
+
Neha Narkhede 2013-03-19, 05:55
+
Jun Rao 2013-03-19, 16:10
+
王国栋 2013-03-20, 07:29
+
王国栋 2013-03-20, 07:34
+
Jun Rao 2013-03-20, 14:29
+
Yang Zhou 2013-03-21, 02:09
+
Yang Zhou 2013-03-21, 02:33
+
Jun Rao 2013-03-21, 04:20
+
王国栋 2013-03-21, 04:43
+
Neha Narkhede 2013-03-21, 05:05
+
王国栋 2013-03-25, 03:58
+
Neha Narkhede 2013-03-25, 04:01
Copy link to this message
-
Re: Kafka throw InvalidMessageException and lost data
Hi Neha,

I am sorry, we couldn't find the detail reason of this bug. But we do
believe that the bug is in java.nio package. Something bad may happen
during sync-producer sends data in byteBuffer or broker receives data from
the channel.

The serialize/deserialize process is right, because we enable valid-message
in producer, the crc checksum is right in client. But the crc checksum is
invalid in the server.
BTW: the bug is only reproduced in high concurrency situation.

Thanks.
On Mon, Mar 25, 2013 at 12:00 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
Guodong Wang
王国栋

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB