Even a trusted tool needs an output limit

Article · Derived state

The tool is internal, authenticated, and read-only. One query returns every matching record, including a few very large text fields.

The model request becomes huge. Logs copy the same result. The final answer uses only the first part, without saying the data was incomplete.

Trust in the tool's identity does not make its output small. Put limits on the result before passing it to the next stage.

Bound several dimensions

Set limits for item count, encoded bytes, individual field size, and total pages or calls. These limits are related, but one does not replace the others.

A Rust output contract can expose whether more data exists:

struct ToolPage<T> {
    items: Vec<T>,
    next_cursor: Option<String>,
    complete: bool,
}

Define complete carefully. It might mean that this query has no more pages, not that the entire external dataset was examined. The next cursor must remain bound to the same authorized query.

The tool-boundary article covers validating generated inputs. The output boundary deserves its own rule because valid inputs can still select a large result.

Enforce the limit before allocation spreads

Ask the data source for bounded pages when it supports them. Count encoded output as you construct the response, before appending it to model context or exporting it to traces.

Do not load an unlimited database result and then call take on the vector. That may protect the model context while leaving the process allocation unbounded.

If one item cannot fit, return a defined oversized-item outcome or a smaller approved representation. Do not return an empty page with the same continuation forever. Advancing past the item silently would hide data instead.

For JSON, construct valid structured output within a budget. Slicing a serialized byte string can break UTF-8 or JSON and conceal the truncation. Rust's String documentation explains its UTF-8 boundary requirements; valid encoding is still only one part of a meaningful tool response.

Tell the model and user about incompleteness

The workflow should know when another page is available and whether its budget permits fetching it. The final answer should not say “all records” when only one bounded page was inspected.

Keep continuation under authorization and a shared operation budget. Pagination should not become an unlimited loop merely because each page is small.

Output limits also affect logging. Emit counts, sizes, and safe references where sufficient rather than copying every result into a trace. A protected tool can still return content supplied by an attacker; size limits do not replace prompt-injection and effect controls.

Test the awkward result

Use many small rows, one oversized row, long multibyte text, and several pages. Assert valid encoding, correct limits, explicit continuation, and termination when the operation budget ends.

Test the complete path into the model request. A gateway may return a bounded page while another middleware adds the full raw response to context or logs.

Some tasks need large datasets. Give them a streaming analysis or export path designed for that scale. A conversational tool should return the amount of information its next step can safely and honestly use.

Stop the read before context grows

The text fixture accepts a reader and a maximum decoded-byte count. It reads at most one byte beyond that count to detect truncation. With an infinite reader and a twelve-byte preview limit, the counter observes thirteen bytes. The returned text is twelve bytes and explicitly partial. It never collects the infinite input and truncates afterward.

Multibyte text gets a valid UTF-8 prefix. A limit landing inside the final character backs up to a valid boundary when the input is truncated. Invalid encoding is refused. An overflowing limit is refused before allocation. A gzip case applies the same policy to decoded bytes and marks the result partial.

Byte limits are not model-token limits, allocator limits, or decompression CPU deadlines. A real model client needs its matching tokenizer if its contract is expressed in tokens. A network reader also needs transport timeouts and a wire-byte policy. The fixture keeps those claims outside the decoded-text result it actually checks.

The page fixture limits a result to three small items, each at most thirty-two bytes, and checks encoded size. It looks ahead by one item to detect continuation. A separate server-held cursor records account scope and offset. Reusing that handle under another account fails. A shared two-call operation budget allows a second page and refuses another, leaving the result explicitly incomplete.

Only the bounded serialized page is appended to a model-context representation. One oversized item is rejected rather than silently skipped. The Rust tests check that representation; they do not send it to a real model or inspect a production tracing exporter.

The cursor store is an in-memory fixture, not a distributed durable pagination service. Its purpose is to make scope, continuation, and the operation budget observable together. Extend those properties to the real source and context builder before claiming an end-to-end bound. A small page is useful only when no parallel path adds the original full result back afterward.