Home | About | Sematext search-lucene.com search-hadoop.com
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> We have an invalid message in a segment, how to deal with it?


Copy link to this message
-
We have an invalid message in a segment, how to deal with it?
Hey guys,

Due to a disk filling up, one of the segments has an invalid message in it.
I have verified this using DumpLogSegments.

How do I deal with this now? the invalid message is causing our Hadoop
Consumer to fail.

Is there a way to remove the invalid message from the segment? Removing the
whole segment causes the broker to fail on startup with an error:

java.lang.IllegalStateException: The following segments don't validate:
<bad-file>, <bad-file+1>

I'm happy losing that file if needs be, but I need to get this broker back
up asap.

--
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
[EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>