Home | About | Sematext search-lucene.com search-hadoop.com
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB
 Search Hadoop and all its subprojects:

Switch to Threaded View
Kafka >> mail # user >> We have an invalid message on a broker, how to deal with it?


Copy link to this message
-
We have an invalid message on a broker, how to deal with it?
Hey guys,

Due to a disk filling up, one of the segments has an invalid message in it.
I have verified this using DumpLogSegments.

How do I deal with this now? the invalid message is causing our Hadoop
Consumer to fail.

Is there a way to remove the invalid message from the segment? Removing the
bad segment causes the broker to fail on startup with an error:

java.lang.IllegalStateException: The following segments don't validate:
<bad-file>, <bad-file+1>

I'm happy losing that file if needs be, but I need to get this broker back
up asap.
Matthew Rathbone
Foursquare | Software Engineer | Server Engineering Team
[EMAIL PROTECTED] | @rathboma <http://twitter.com/rathboma> |
4sq<http://foursquare.com/rathboma>

 
NEW: Monitor These Apps!
elasticsearch, apache solr, apache hbase, hadoop, redis, casssandra, amazon cloudwatch, mysql, memcached, apache kafka, apache zookeeper, apache storm, ubuntu, centOS, red hat, debian, puppet labs, java, senseiDB