Consider what happens when a member asks an assistant for their recent transactions. The member is authenticated to the banking channel, and the assistant then calls a transaction-history function on their behalf. The question that decides whether the system is safe is whose identity that function call carries when it arrives at the transaction API.
Two architectures answer it differently, and they are indistinguishable from outside the system, because both produce the right transactions for the right member on an ordinary request.
In the first, the agent acts on behalf of the member. The member’s session token, or a token derived from it, travels with the tool call, and the transaction API returns only what that member is entitled to see. Where the model is manipulated into requesting someone else’s transactions, the API refuses, because it evaluates the token rather than taking the model’s word for who is asking.
In the second, the agent acts as itself rather than for anyone. It holds a service identity with read access across all members, and it passes a member identifier as a parameter, which the model selected from the conversation. Where the model is manipulated into passing a different identifier, the API returns that member’s data, because from the API’s position the request arrived from a fully authorized identity and is indistinguishable from a legitimate one.
The second architecture is easier to build, because one identity is provisioned once and every channel uses it, and it behaves correctly on ordinary requests because the model selects the right identifier from a clear conversation. What it does is convert an authorization decision into a natural-language inference, so the boundary that protects one member’s data from another is the model’s reading of the conversation rather than a check performed by the system that holds the data.
Two situations turn the second design from a weakness into an exposure that can be acted on today.
The first is a shared channel, where the same agent serves two populations. The same agent, backed by the same service identity, is exposed inside the authenticated application and on a public page where no authentication has taken place. The agent’s authority does not change between the two, because authority is a property of the identity and the identity is the same, so only the surrounding context differs. What stands between an anonymous visitor and a member’s data is then the model’s judgment about who it is speaking to, and that judgment is formed from text the visitor supplied.
The second is untrusted content reaching the context window, where the agent reads material somebody outside the review can write. Where any document, ticket, email, or knowledge-base page the agent retrieves can contain instructions, whoever can write to that source can attempt to redirect the tool call. Identity binding is the control that makes the attempt fail closed, because a member-scoped token returns nothing for a record the member is not entitled to, whatever the model was persuaded to request.
Ask for the channel configuration first, meaning every surface on which the agent is exposed and the authentication state of each. Where that information is not held in one place, the absence is itself informative, because it means the question of where the agent is reachable from has no owner. Then establish, for each channel, which identity carries the tool call.
Then test on the public surface as an unauthenticated visitor, by asking for something only an authenticated member should receive. Evidence obtained that way ends an argument about likelihood, because it demonstrates the condition rather than describing it.
For the finding, a sentence saying the agent may disclose member data invites a discussion about probability that has no resolution. Write the mechanism instead: authorization for member data is determined by a parameter the model selects rather than by the identity of the requester, and the same agent identity serves the authenticated and the anonymous channel. A mechanism tells management what to change, and a statement of possibility does not.
The remediation that follows is per-request identity propagation, in which the tool call carries the end user’s token and the target system enforces entitlement independently of anything the model produced. A revised system prompt does not address it, because the prompt is the layer the manipulation acts on, and an output filter does not address it either, because by the time the filter runs the data has already been read.
This brief is one of ten behind Module 1, The tool surface, a free guided walkthrough of an AI agent assessment at a credit union. The same engagement can be run unassisted, with the check questions above put to you against evidence rather than against a description.
NexNith advises boards and audit committees on exactly this work. How we work