Log the tool decision without copying the customer's document

Article · Through the layers

The tool was denied. To investigate, the team opens the trace and finds the full customer document, the generated arguments, and a provider response containing private notes.

The trace explains the decision. It also created another copy of data that the system now has to protect and eventually delete.

Start by asking which facts the investigation needs. Often they are much smaller than the content that happened to pass through the tool.

Give the decision its own event

A useful audit event can identify the operation, actor scope, requested capability, policy version, decision, and effect outcome. Keep the vocabulary bounded.

For example, a Rust event model could begin like this:

pub struct Audit {
    pub operation_id: String,
    pub actor_reference: String,
    pub tool_name: &'static str,
    pub policy_version: u64,
    pub decision: &'static str,
    pub outcome: &'static str,
    pub request_id: String,
}

Use server-issued references where possible and apply size limits to strings. An actor reference is still potentially personal data; it needs access and retention rules even without a full document attached.

Decision and outcome are separate. An allowed operation can fail at the provider. A denied operation should remain not started. That distinction helps identify enforcement that runs too late.

The privacy-aware evaluation-data article explains how production observations can become safer test material. A small operational event is a better starting point than indiscriminate prompt capture.

Errors are another output channel

A carefully designed success log can be undermined by a debug-formatted error that includes the request body. Check wrapped errors, provider messages, retry logs, and panic reports.

OWASP's logging guidance discusses sensitive data and safe event collection. Allow only the fields you intend to log. A generic serializer cannot decide which values your product should keep private.

Do not put untrusted text directly into a log format that can create extra lines or fake fields. Structured logging helps, but consumers still need to handle values safely.

If content capture is necessary for a specific investigation, make it a separate controlled path with a reason, restricted access, short retention, and appropriate redaction. Do not make every routine trace a document archive.

Check with canaries

Place unique synthetic secrets in the model arguments, retrieved document, provider error, and nested error source. Trigger allowed, denied, and failed calls. Collect every event the workflow emits and search for those canaries in CI.

Also assert that the useful metadata remains. A test that makes logging completely silent protects privacy by destroying operational evidence. The goal is a bounded event that still explains the decision.

Review the actual exporter and sink configuration. A local event formatter cannot prove that an instrumentation layer is not separately recording full request bodies.

Keep the event useful after a policy change

Record a policy version and operation identity so an investigation can explain the rule applied at the time. Avoid storing an entire permission document when a stable reference and version are enough.

The useful log says who requested which capability, what the server decided, and what happened next. Most of the time, it can answer those questions without repeating the customer's document.

Inspect the emitted sink with synthetic canaries

The reproduction emits a small JSON audit record for each invocation. It retains operation and actor references, a known tool name, policy version, decision, outcome, and request reference. Unknown tool names become a fixed “unknown” label rather than copying arbitrary input into the event vocabulary.

The test puts a canary in a preview note, another in an unknown tool name, and more in a provider error with a nested source description. It also sends oversized input and a request reference containing a newline. The gateway records four events covering success, malformed or denied input, and an allowed transport failure.

Each serialized event must be one line and smaller than one kilobyte. None may contain the document, argument, tool-name, provider-error, nested-source, or injected-request canaries. The operation reference and useful outcome labels must remain. Silence would not satisfy this test.

The failure adapter returns a typed transport outcome. It does not serialize the provider's raw diagnostic string. The safe event formatter receives only the audit fields, so it cannot accidentally walk an error chain that was never passed to it. This reduces the amount of content a routine log path can expose.

The Rust sink test is intentionally local to this gateway's emitted events. It does not inspect a production tracing agent, reverse proxy, crash collector, or external log exporter. Those layers can still capture bodies independently and need their own review with the same synthetic markers.

Keep references bounded and treat them as potentially sensitive metadata. A safe actor reference may still identify a person to an authorized investigator. Access, retention, and deletion rules remain relevant. The practical gain is that normal diagnosis can answer which policy allowed or refused which capability without storing every document that passed through the request.