Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Plain View
Kafka >> mail # user >> We have an invalid message in a segment, how to deal with it?


Copy link to this message
-
We have an invalid message in a segment, how to deal with it?
Hey guys,

Due to a disk filling up, one of the segments has an invalid message in it.
I have verified this using DumpLogSegments.

How do I deal with this now? the invalid message is causing our Hadoop
Consumer to fail.

Is there a way to remove the invalid message from the segment? Removing the
whole segment causes the broker to fail on startup with an error:

java.lang.IllegalStateException: The following segments don't validate:
<bad-file>, <bad-file+1>

I'm happy losing that file if needs be, but I need to get this broker back
up asap.

--
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
[EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>

 
+
Jay Kreps 2013-04-13, 18:28
+
Jay Kreps 2013-04-13, 18:32
+
Matthew Rathbone 2013-04-13, 18:51
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB