Agent audit logs and regression checks a small team can maintain

A transcript is not an execution record

An assistant can say an action was blocked while a poorly wired handler still performs it. It can also say a message was sent when a downstream service rejected it. For an investigation, neither sentence is enough. The application needs evidence from the decision point and the execution adapter, connected to the initiating request.

OWASP's Logging Cheat Sheet recommends security event records with sufficient context, restricted access, and careful exclusion of tokens, passwords, and sensitive data. The event model below is an original example for a small workflow. It is a proposed design, not a claim that a deployed Boundary feature collects these events.

An illustrative event pair

Imagine a test agent proposing a draft-message send. Assign an opaque request identifier in the trusted application. Log the policy decision, reason code, policy revision, authenticated actor reference, action type, and a minimized target reference. If execution is permitted, the adapter records its result under the same request identifier. Use UTC timestamps from the component generating each event and record ingestion time separately where ordering matters.

{
  "request_id": "synthetic-run-42",
  "event": "policy_decision",
  "action": "message.send",
  "decision": "approval_required",
  "reason_code": "external_send",
  "policy_revision": "example-policy-3"
}
{
  "request_id": "synthetic-run-42",
  "event": "execution_status",
  "status": "not_started"
}

When an approved action executes, add a result such as succeeded, failed, or unknown from the adapter. An ambiguous network timeout should stay unknown until reconciled with the receiving service. Treat retry eligibility as a separate decision; a repeated request can otherwise create duplicate sends or charges.

Do not let model output populate authoritative fields such as authenticated actor, approval issuer, policy revision, or success state. The model can propose an action and explanatory text; trusted components supply the security facts. A tool response that merely repeats the model's requested outcome is not independent evidence.

Minimize the evidence without losing the question

Write down the investigation questions before adding fields. To answer “did the application allow a send without approval?”, you need the decision, relevant approval reference, and adapter outcome. You rarely need the entire message body in every event. Where exact arguments must be checked, use a protected action record with tightly controlled access and a reference from the ordinary event stream.

Choose retention by purpose and obligation, then implement deletion and access control for each store. Bound text lengths and safely encode untrusted values so a crafted newline or HTML fragment cannot impersonate another event in the viewer. Limit who may query, export, or change logs. A log store with broad read access can become another disclosure path.

Review what your telemetry SDK sends on exceptions. Disable capture of authorization headers and raw payloads where they are unnecessary. Use fake markers in a development fixture to check the configured collection path rather than assuming a dashboard masking setting changes the underlying stored data.

A regression matrix built around outcomes

OWASP's AI application testing guidance calls for testing the complete application, maintaining a regression corpus, recording configuration, and checking high-impact failures separately from averages. Apply that idea to a short set of behaviors you can review.

Use recording adapters that cannot send, delete, pay, or reach production. Set the expected outcomes before looking at model responses. Assert on attempted tool calls and resulting fixture state, not just a judge's opinion of the final answer. Keep a benign counterpart so an always-deny implementation does not appear successful.

Use failures to make the next release smaller

When a case fails, retain the smallest synthetic reproduction, identify the missing check, and add the fixed case to the routine suite. Record application commit, policy revision, model identifier, prompt revision, tool manifest, and fixture version. If the provider exposes an immutable model snapshot, record it; otherwise note that the model identifier may not freeze behavior.

Run repeated trials where model variability matters and report the number of trials and observed failures honestly. A passing sample only supports a statement about those cases and that configuration. Before a release, separately review high-impact failures, false denials on normal tasks, and unknown execution outcomes. Assign someone to investigate alerts and make the disable path practical; logging without an owner does not create response capacity.

Scope and limits

These examples are a proposed baseline for a small AI application. They are not a security certification, an exhaustive adversarial evaluation, or evidence of a prevention rate. Logs can be incomplete or tampered with; protect their integrity and record collection gaps. Use authorized isolated environments for adversarial checks and do not probe unrelated services.

Boundary is a prelaunch commercial model router. Planned request-path policy checks do not imply an implemented agent audit platform or control over file access, external messages, payments, deletion, or bypass routes. The broader operational guidance here applies to systems you own and can configure.