A supervisor approval step on refunds was removed in March, and no change record exists. The explanation, when asked for, is neither evasive nor unreasonable. The queue was backing up, average handle time was the metric the team was measured on, the step was slowing the queue, and it was a toggle on a settings screen. Somebody with legitimate access to that screen turned it off. No code was deployed, so no deployment occurred, so nothing entered the change process.
The condition is under-tested because of a mismatch between how change control is defined and how these platforms are configured, and the mismatch is structural rather than a lapse by anyone involved.
Change control was built around code, at a time when a change to system behaviour required one. A deployment triggers it and the pipeline is the point at which it is enforced. Agent platforms have moved the consequential decisions out of code and into configuration: the system instructions, the tool definitions, the approval thresholds, the human-in-the-loop toggles, the retrieval sources, and the routing rules. Each of those is editable in a console, by product and operations staff, with immediate effect and no pipeline between the edit and production.
The controls that matter most therefore sit outside the process designed to govern change. The people making the changes are not circumventing anything, because they were given a settings screen and told it was theirs to operate.
A control that was never implemented appears in a design review, because the design review compares the design against a requirement. A control that was implemented, evidenced at implementation, and switched off eight months later appears in neither the design review nor the walkthrough. The design review predates the change, and the walkthrough demonstrates the current screen, on which the step is absent rather than visibly missing.
It appears in two places, and both have to be asked for by name. The first is the platform’s configuration change history, where the platform keeps one and where it is asked for by name. The second is the transaction record, where the absence of the control leaves a trace, in transactions that should have required approval and did not, or amounts that should have been rejected and were not. The transaction record is cheaper to obtain and harder to argue with, because it is an instance of the system acting.
Ask for the platform’s configuration audit trail by name and for the full period under review, rather than for the change advisory board records. The board records will show a small number of deployments and none of the configuration changes, because the configuration changes never reached the board.
Some platforms keep no configuration history, and others retain it for a limited window. Where the history is absent, that absence is a finding in its own right and a graver one than the specific change being sought, because it means no configuration change to this system can be evidenced at all.
Then establish who holds edit rights on the console, and compare that population against the population that goes through change control.
Two findings live here, and separating them matters because they have different owners and different remedies.
The narrow finding is that an approval control on financial transactions was disabled without authorization or record, and that transactions have since occurred which the control was designed to prevent.
The broad finding is the one that changes anything: the change management process does not extend to the platform configuration where this system’s controls are implemented. Addressing only the narrow finding restores one toggle, and the condition returns with the next configuration change. Addressing the broad one closes the class, because changes to agent behaviour, tool definitions, thresholds, and human-in-the-loop steps then require an approver, produce a record, and are reconciled periodically against the approved state.
This brief is one of ten behind Module 1, The tool surface, a free guided walkthrough of an AI agent assessment at a credit union. The same engagement can be run unassisted, with the check questions above put to you against evidence rather than against a description.
NexNith advises boards and audit committees on exactly this work. How we work