Kafka crashed __consumer_offsets compaction —
Disclaimer: this post is about an ancient (0.10.0.1) version of Kafka and the bugs described have since been fixed.
I’ve got a (legacy) kafka cluster that is still being used, pending an upgrade that takes a bit more time than expected. This kafka cluster is still used for many workloads, we have plenty of experience with it and it runs flawlessly. Or so that’s what we thought.
At some point, we lost a broker.