#metadata
Posts
-
Digging into Kafka's Source 10: KRaft and Metadata — How the Controller Remembers the Cluster
The cluster's ledger — topics, partitions, leaders, the list of brokers — is stored as records in __cluster_metadata, an ordinary Kafka log, and the controllers keep that log together with Raft. We follow it through the source: the roles in KafkaRaftClient (Prospective, Candidate, Leader, Follower) with Vote and Fetch, a high watermark set by a majority of voters, a QuorumController that runs on a single event thread and applies records to memory before they commit (and rolls them back with SnapshotRegistry), the MetadataLoader that turns records into MetadataDelta and MetadataImage on the broker, and heartbeats and fencing — checked with kafka-dump-log, kafka-metadata-quorum and a three-controller election.