Posts tagged "what survives a crash"

SQLite synchronous, PostgreSQL synchronous_commit, MySQL innodb_flush_log_at_trx_commit, Redis appendfsync: each one decides how much a crash may lose. I put them in one table, with a column for each kind of crash. Only SQLite is measured here; the others are their documentation.

In WAL mode, a SQLite commit with synchronous=FULL is one frame and one fdatasync, and with NORMAL it is no sync at all. I followed both with strace on my laptop, measured them, and quote what the documentation promises after an application crash and after a power loss.

One INSERT in SQLite's default rollback journal mode makes four sync calls on my laptop. I followed every write with strace, compared FULL, EXTRA, NORMAL and OFF, and found the one directory sync that FULL leaves out, as the SQLite documentation says.

My program got EIO from fdatasync, called it again and got ok. In 2018 PostgreSQL learned why that second ok can be a lie on Linux. I tell the documented story, then inject fsync errors into my store with an LD_PRELOAD shim, and I say what the shim cannot show.

A recovery that writes files can crash too, and then a second recovery must give the same result. I killed my store 500 times during its own recovery: one design always recovered correctly, the other doubled a counter 53 times. The fix is one number in the snapshot.