Kafka, mail # user - We have an invalid message on a broker, how to deal with it? - 2013-04-13, 17:03
Solr & Elasticsearch trainings in New York & San Francisco [more info][hide]
 Search Hadoop and all its subprojects:

Switch to Threaded View
Copy link to this message
-
We have an invalid message on a broker, how to deal with it?
Hey guys,

Due to a disk filling up, one of the segments has an invalid message in it.
I have verified this using DumpLogSegments.

How do I deal with this now? the invalid message is causing our Hadoop
Consumer to fail.

Is there a way to remove the invalid message from the segment? Removing the
bad segment causes the broker to fail on startup with an error:

java.lang.IllegalStateException: The following segments don't validate:
<bad-file>, <bad-file+1>

I'm happy losing that file if needs be, but I need to get this broker back
up asap.
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
[EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB