Posts tagged "data integration"

An SFTP folder is not a queue. Reliable imports need a ready-file contract, stable file identity, durable claims, idempotent processing, and reconciliation.

When all tenants consume one provider quota, local retries and per-worker limits are not enough. Use a global budget with fair tenant admission and feedback.

A shared FIFO queue lets one large customer delay everyone else. Schedule integration work by tenant, cost, and age while preserving useful spare capacity.

An adapter contract and test suite for translating provider pagination, identity, nulls, versions, errors, retries, deletion, and capabilities—not only field names.

A concurrent refresh design using local single-flight work, database versions, conditional writes, and recovery rules for providers that rotate refresh tokens.