Crashes, retries, cancellation, partial progress, ownership, recovery and idempotency.
3 parts
Five async Rust problems reproduced in one small tokio program: an oversized future, a future that is not Send, a blocking call, a select! loop that loses data, and a shutdown that hangs. Then the cost of async fn through dyn Trait, and cleanup when Drop cannot await.
Evidence: experiments/rust-atlas/async-symptoms/
A checkpoint is a claim about durable effects, not the last page fetched. Compare commit protocols and crash points so recovery repeats work instead of losing it.
A paginated import should resume from a committed cursor and stable item identities, not from a human page number that may no longer describe the same data.
Async cancellation usually means dropping a future after Pending. See how select loops lose partial progress and how to design restartable operations.
A concurrent refresh design using local single-flight work, database versions, conditional writes, and recovery rules for providers that rotate refresh tokens.
Agent retries must reuse one logical action identity and reconcile uncertain effects. An effect ledger makes autonomy bounded, inspectable, and recoverable.
A crash-safe checkpoint design for incremental API synchronization, including atomic progress, cursor invalidation, poison records, leases, and full-sync recovery.
A deduplication TTL must cover every supported path by which an old message can return. Derive the window from retries, replays, outages, and redrives.
A database inbox turns at-least-once delivery into one committed business effect by claiming message identity in the same transaction as domain changes.
Large API batches reduce request overhead until one bad record makes good records fail with it. A measured way to find the useful batch size and retry unit.
A bounded API retry policy that combines error classification, Retry-After, full-jitter backoff, operation deadlines, idempotency, and a shared retry budget.
An SFTP folder is not a queue. Reliable imports need a ready-file contract, stable file identity, durable claims, idempotent processing, and reconciliation.
A decision and failure matrix for combining low-latency webhooks with polling and reconciliation so an integration can recover from missed, duplicate, or reordered events.
Webhook verification middleware and tests that authenticate the exact request bytes, reject stale signed deliveries, support key rotation, and deduplicate effects durably.
A synchronization run rarely fails as one clean unit. Define progress, safe retries, checkpoints, and recovery so partial success becomes an ordinary state.
A useful fake provider controls time, pagination, throttling, malformed data, and ambiguous writes. Test the integration protocol, not only JSON decoding.
A measured additive-increase, multiplicative-decrease controller for finding useful API concurrency while respecting throttling, latency, retries, and tenant isolation.
When all tenants consume one provider quota, local retries and per-worker limits are not enough. Use a global budget with fair tenant admission and feedback.
Older, introductory or narrower pieces on the same subject.