Mehdi Akiki
Writing / Series

Reliable Data Integrations

Synchronization, events, schemas, and recovery

Engineering patterns for integrations and distributed data flows that must stay understandable when APIs change, deliveries repeat, and parts of the system fail.

The model behind the series

Connecting two systems is easy in the successful case. The durable design appears in the less comfortable cases: duplicate delivery, partial progress, conflicting changes, disappearing records, schema drift, and a replay that happens months later.

This series treats integration as a systems problem. It develops explicit models for identity, ownership, ordering, recovery, and observability, then shows how those models guide implementation.

Core principles

01

Model identity and ownership first

A synchronization flow needs explicit answers about which records correspond and which system may change each field.

02

Design the replay before the happy path

Retries, idempotency, checkpoints, tombstones, and reconciliation are normal operation, not exceptional add-ons.

03

Translate at system boundaries

Stable internal models contain provider-specific schemas and keep external change from spreading through the product.

Start here

Entry points into this subject, ordered as a reading path.

01

How to Keep Two Systems in Sync

When you sync two systems with webhooks, the copy slowly becomes wrong: some events are lost, some are duplicated, some arrive in the wrong order. Reconciliation is the way to make the two systems agree again. Here is how to design an API for it.

03

gRPC from the Ground Up: The Real Parsing Stack

What actually happens between bytes arriving on a socket and your gRPC handler running? This article walks one real request through every parsing layer — TCP, HTTP/2 frames, HPACK, gRPC envelope, protobuf — so the magic disappears.

Investigations

Multi-part work with the code and measurements that produced its evidence.

20 parts

What survives a crash

Twenty parts on what really reaches the disk: write and the page cache, the cost of fsync, saving a file with rename, the directory fsync, torn writes, a crash simulator built from strace output, a small write-ahead log with checkpoints and recovery, SQLite journal and WAL modes, group commit, mmap, checksums and crash testing. Small programs in Rust, Python and C, with one script that reproduces every output.

10 parts

Following a TCP connection

Ten parts that follow one TCP connection on loopback, from connect() to a connection pool: the handshake, the accept queue, Nagle and delayed ACK, the byte stream, full socket buffers, what close() sends, TIME_WAIT, getaddrinfo and the cost of TLS. Only what ss, the kernel counters, timings and system calls show, with one script that reproduces every output.

Articles

Through the layers

Put two tenants in your authorization tests

A one-tenant fixture cannot show cross-tenant access bugs. Exercise reads, writes, and reused database connections with two independent customer scopes.

Through the layers

Decide what happens when the queue is full

A bounded worker queue needs an admission policy. Choose between waiting, rejecting, and durable acceptance, and tell callers which promise they received.

Through the layers

Don't fix a flaky test with a longer sleep

Replace timing guesses with observable readiness and controlled time. Keep real transport deadlines separate from a deterministic retry-policy test.

Through the layers

A Rust tool for an existing Db2 database

Connect a bounded Rust export tool to Db2 LUW with a real IBM ODBC driver, then check the server version, null values and truncated fields.

Through the layers

A SQL Server read waited behind an update

Reproduce a blocked SQL Server read from Rust, check READ_COMMITTED_SNAPSHOT and distinguish lock timeout 1222 from an application timeout.

Derived state

Read Old MariaDB Row Versions From Rust

Use a system-versioned table to read values after update and deletion, while keeping business time, access and backups explicit.

Interrupted execution

Two Webhook Deliveries Need One Database Claim

Two webhook handlers can both pass an existence check. Use a unique database claim, bind it to the payload, and commit it with the business change.

Interrupted execution

Use a Separate Database Identity for Migrations

Separate schema ownership from normal application access. Test that the runtime account can do its work and cannot change the tables or security policies.

Interrupted execution

A SQLite Import Can Block Normal Saves

Keep an interactive SQLite importer resumable and bounded. Commit batches with their progress and give foreground writes an explicit busy policy.

Interrupted execution

Choose one layer to own retries

Nested SDK, transport, and worker retries can multiply attempts. Give one logical operation an effective retry budget and a deadline that every layer respects.

Interrupted execution

A bulk response must say which items succeeded

Give every bulk item a stable identity and an explicit outcome. Distinguish atomic batches, independent writes, and unresolved results before designing retries.

Interrupted execution

Give repair scripts a preview before they write

Separate repair planning from application. Show the selected records and proposed changes, then reject stale versions instead of applying an old preview blindly.

Interrupted execution

Retry the Whole PostgreSQL Transaction from Rust

Force a serialization conflict with two SQLx connections and check that a bounded retry opens a new transaction and repeats the reads that chose the write.

Interrupted execution

Loading a batch into SQL Server from Rust

Use Tiberius bulk copy with checked constraints and explicit transactions, then verify rollback, committed rows and rejected data separately.

Interrupted execution

Resuming a Paginated Import After Page 417 Fails

A paginated import should resume from a committed cursor and stable item identities, not from a human page number that may no longer describe the same data.

Interrupted execution

Preventing an OAuth Token Refresh Stampede

A concurrent refresh design using local single-flight work, database versions, conditional writes, and recovery rules for providers that rotate refresh tokens.

Interrupted execution

Webhook Signatures Need Replay Protection Too

Webhook verification middleware and tests that authenticate the exact request bytes, reject stale signed deliveries, support key rotation, and deduplicate effects durably.

Interrupted execution

Sharing One Provider Quota Across Many Customers

When all tenants consume one provider quota, local retries and per-worker limits are not enough. Use a global budget with fair tenant admission and feedback.

Measurement

A test should fail for the reason you meant

An is_err assertion can pass because the database is unreachable. Check the intended error category and the stored outcome so unrelated failures cannot make a negative test green.

Shorter pieces and reference

Older, introductory or narrower pieces on the same subject.

Engineering context

When an integration has to survive production

I design data and integration systems around explicit contracts, observable failure, safe recovery, and the realities of changing external APIs.