#crash-recovery
Posts
-
Digging into Kafka's Source 7: Log Storage — Segments, Indexes, Retention and Compaction
When does a file like 00000000000000003830.log appear in a partition directory, how is one offset turned into a byte position in that file, who deletes a segment once retention.ms has passed, and how does cleanup.policy=compact sweep away old values of the same key? We follow UnifiedLog, LocalLog, LogSegment, OffsetIndex and LogCleaner in the source, and check with examples the five roll conditions, the sparse index that writes one entry per 4 KB and lives in mmap, the two-step delete through a .deleted rename, the SkimpyOffsetMap built from MD5 hashes of keys, and the recovery point used when a broker comes back from a crash.
-
Digging into MariaDB's Source 12: Two Logs — the Redo Log, the Binlog and Crash Recovery, and Wrapping Up
What it means for data to be on disk when COMMIT returns. We follow InnoDB's redo log (log_sys, LSNs, mtr_t::commit, log_write_up_to) and the spot where innodb_flush_log_at_trx_commit splits, the checkpoint and the layout of ib_logfile0, crash recovery after kill -9 that rolls forward with redo and back with undo, the server-level binlog and its events, and the two-phase and group commit that tie the two logs together by XID. Using DBUG crash points we kill the server in both gaps of the two-phase commit and watch what recovery does, then wrap up all twelve parts of the series.