What sensitive data should an AI app send to a model?

Start with a field-level decision

An AI feature can need the meaning of a support ticket without needing the sender's phone number, account token, or complete billing history. Make the request payload visible in a controlled development fixture and assign a reason to each field. The useful question is “what decision does this field enable?” A field with no answer is a candidate for removal.

OWASP's sensitive-information disclosure guidance covers credentials, personal information, and confidential business material, and discusses access controls and tokenization/redaction. The example below turns those principles into an original small-team review exercise. It uses invented records and makes no claim of measured leak prevention.

A synthetic ticket, reduced before inference

Suppose a classifier sorts support tickets into billing, technical, and other. Its test ticket contains a fabricated contact address, an internal account identifier, and this message: “My scheduled report failed after I changed its timezone.” The classifier needs the issue text. It probably does not need the contact address or account identifier.

Input fixture:
  ticket_id: synthetic-204
  contact: operator@example.com
  account_id: fake-account-9
  message: My scheduled report failed after I changed its timezone.

Candidate model payload:
  task: Classify the issue as billing, technical, or other.
  message: My scheduled report failed after I changed its timezone.

Build the candidate payload from an explicit set of fields. Avoid serializing an entire ticket object and hoping a redactor catches everything later. Keep the lookup needed to deliver the result inside your application. If the model needs to distinguish multiple tickets, use a short synthetic request reference and resolve it in trusted application code.

Now consider free text: the customer may paste a credential into the message. A field allowlist does not remove sensitive values embedded in an allowed field. Add checks for the specific data classes you have identified, decide whether to block, mask, or request a sanitized submission, and preserve a useful error state. Do not silently send the original body when the checker times out.

Distinguish data-handling questions

Record separate answers for provider, model, endpoint, region where applicable, request storage, application-state storage, abuse monitoring, training use, and deletion behavior. Check feature exceptions for files, caches, batch processing, code execution, or stored conversations. The same provider name can cover multiple arrangements.

For example, Anthropic's API retention documentation distinguishes processors/platforms and feature eligibility, and states that zero data retention is enabled per organization. This illustrates why a general account label is insufficient. Read the current official documentation and your applicable agreement for the exact route you use; this article does not assert that a particular account qualifies for any arrangement.

Keep an application inventory row for every route: “feature → allowed data classes → provider/endpoint → relevant settings evidence → owner → review date.” Update it when a routing fallback, model, or API feature changes. If a fallback cannot accept the original data class, fail the request or transform it under a reviewed policy before retrying.

A small redaction and routing test set

1. Known marker: place a deliberately fake secret marker in an allowed message field. Assert the configured block or mask result and inspect the outbound request stub.
2. Unexpected field: attach a new internal_notes field to the ticket fixture. Verify it never enters the constructed payload.
3. Unrecognized sensitive prose: add an invented confidential acquisition plan that matches no regex. Document that the pattern checker cannot establish its safety; use this case to test the workflow's data policy.
4. Fallback route: simulate the primary endpoint being unavailable. Verify the fallback applies its own permitted-data decision.
5. Checker failure: simulate timeout and malformed output. Confirm the application follows its explicit failure policy.

Use a local recording stub or your isolated authorized test environment. These are suggested checks, not tests conducted against a live vendor. Inspect logs and error reporting too: removing a marker from the model request while leaving it in an exception trace still creates another sensitive copy.

Handle reversibility and retention

If placeholders are necessary, store any mapping where ordinary application authorization and deletion controls apply. A placeholder is still linkable when a mapping exists. A deterministic hash of a guessable value may also be linkable, so do not describe hashing alone as anonymization. Restrict access to the map and decide when it expires.

After a suspected disclosure, preserve a minimal event record, identify the routes and stored copies involved, restrict the affected feature as needed, and follow your incident process. Changing a future request template does not recall a payload already processed. Avoid turning debugging into a second disclosure by copying raw customer prompts into chat or issue trackers.

Scope and limits

Pattern matching has false negatives and false positives; it cannot reliably identify every sensitive fact, encoding, attachment, or contextual disclosure. This workflow is a technical starting point, not a compliance determination. Teams handling regulated or contract-restricted data need requirements appropriate to those obligations.

Boundary is a prelaunch commercial model router with request checks for defined sensitive patterns and model/provider policies in the planning stage. This article does not promise a complete PII detector, automatic deletion of provider copies, or enforcement over tool calls that bypass a router.