Kafka, mail # user - Re: Kafka throw InvalidMessageException and lost data - 2013-03-25, 04:23
 Search Hadoop and all its subprojects:

Switch to Plain View
+
Helin Xiang 2013-03-19, 03:22
+
Jun Rao 2013-03-19, 04:58
+
Helin Xiang 2013-03-19, 05:13
+
Neha Narkhede 2013-03-19, 05:55
+
Jun Rao 2013-03-19, 16:10
+
王国栋 2013-03-20, 07:29
+
王国栋 2013-03-20, 07:34
+
Jun Rao 2013-03-20, 14:29
+
Yang Zhou 2013-03-21, 02:09
+
Yang Zhou 2013-03-21, 02:33
+
Jun Rao 2013-03-21, 04:20
+
王国栋 2013-03-21, 04:43
+
Neha Narkhede 2013-03-21, 05:05
+
王国栋 2013-03-25, 03:58
+
Neha Narkhede 2013-03-25, 04:01
Copy link to this message
-
Re: Kafka throw InvalidMessageException and lost data
Hi Neha,

I am sorry, we couldn't find the detail reason of this bug. But we do
believe that the bug is in java.nio package. Something bad may happen
during sync-producer sends data in byteBuffer or broker receives data from
the channel.

The serialize/deserialize process is right, because we enable valid-message
in producer, the crc checksum is right in client. But the crc checksum is
invalid in the server.
BTW: the bug is only reproduced in high concurrency situation.

Thanks.
On Mon, Mar 25, 2013 at 12:00 PM, Neha Narkhede <[EMAIL PROTECTED]>wrote:
Guodong Wang
王国栋

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB