Set a byte limit before collecting a response body

The service expects a small JSON document. It collects the entire response into a vector and parses it afterward.

That is convenient while the upstream behaves. A bad response, a proxy error, or a hostile endpoint can turn the same code into a large allocation.

Adding a buffered reader does not set a total input limit. It changes how reads are grouped. The collection can still grow until the input ends or memory runs out.

Check before you collect everything

For a synchronous Rust reader, a small helper can read at most one byte beyond the accepted limit. That extra byte distinguishes an input exactly at the limit from an oversized one:

use std::io::{self, Read};

pub fn read_bounded(reader: impl Read, max: usize) -> io::Result<Vec<u8>> {
    let probe = max.checked_add(1)
        .ok_or_else(|| io::Error::other("limit is too large"))?;
    let mut bytes = Vec::new();
    reader.take(probe as u64).read_to_end(&mut bytes)?;
    if bytes.len() > max {
        return Err(io::Error::new(io::ErrorKind::InvalidData, "response exceeds byte limit"));
    }
    Ok(bytes)
}

Choose a modest application limit; this is not an allocator for arbitrary user-supplied sizes. On normal supported Rust targets, the conversion covers a usize limit that already passed the checked addition.

Rust's Read::take caps the bytes available through that adapter. The helper rejects overflow instead of returning a silently truncated document.

The equivalent asynchronous implementation should count chunks before extending the collected body. Reject a chunk that would exceed the remaining budget, with checked arithmetic. Do not first collect the whole async body and call this helper on the resulting slice.

A header is useful, but not enough

A declared Content-Length can support an early refusal. It is not your only enforcement point. The client may receive a streamed response, and automatic decompression can make the decoded body larger than its wire representation.

Decide which limit protects which stage. A wire-byte limit protects transfer volume. A decoded-byte limit protects the data passed to the parser. A parser may need its own limits for nesting, field counts, or unusually large individual values.

Also set a deadline. A reader that sends one byte at a time can stay below the byte budget while occupying a request for too long. Size, time, and concurrent requests are separate resources.

The BufReader explanation covers buffering. Keep that mechanism separate from your application's acceptance rule.

Exercise the boundary values

Use inputs just below the limit, exactly at it, and one byte above it. Include an empty body and an input that fails partway through reading. The error should remain an error; do not parse a partial prefix as a successful response.

For an HTTP adapter, add a body with no declared size and a compressed body that expands beyond the decoded limit. Check that the adapter stops consuming rather than merely reporting the excess afterward.

Some workflows genuinely process large files. They need streaming or bounded storage rather than a very large in-memory cap. For the small JSON endpoint, the useful improvement is simpler: make the expected size a rule the reader enforces before the parser sees the body.

Apply the reader to a real HTTP response

The complete bounded input adapter uses reqwest's blocking reader inside a blocking task. Its loopback HTTP fixture sends chunked responses without Content-Length. Sixteen decoded bytes are accepted at a sixteen-byte limit; seventeen are rejected. The collector enforces the rule even when there was no declared size to trust.

The compressed comparison has two bounds. The adapter first accepts at most 65,536 encoded bytes, then feeds those bounded bytes into flate2's gzip decoder and applies the decoded limit. A response expanding to a thousand repeated bytes must be rejected at the decoded boundary. Unsupported content codings are errors. The multi-member decoder also rejects a concatenation whose first member is small but whose later member exceeds the decoded budget. Another compression protocol needs its own decoding contract.

A stalled response exercises the client's 300-millisecond deadline. That number keeps the lab bounded. It is not a recommended service timeout or a latency result. The network deadline also does not make synchronous decompression preemptible. CPU-heavy parsing or decoding needs its own admission and execution policy.

The accepted byte count is not a whole-process memory cap. Vec can reserve spare capacity, the HTTP implementation can buffer protocol data, and the decoder has internal state. The fixture protects the size of the collected input handed onward. It does not claim that every network or allocator buffer is at most sixteen bytes.

Keep the synchronous helper's boundary and source-error tests as well. They check empty input, exact input, overflow, and a failing reader. An HTTP demonstration should add to those small checks rather than hide their error semantics.

The decoder implementation uses flate2 1.1.5. The budget applies across the decoded members. The encoded bound still applies to the compressed input retained before decoding.