The AI Was Wrong. The Human Clicked Approve.

A pause does not protect human agency unless it reveals the risk and gives the user power to change the outcome.

Imagine a fire breaking out in a crowded office tower.

An AI evacuation system analyses the building map, smoke sensors, and movement of people. It recommends Stairwell B. Before the route is announced, the controller must complete four checks:

• Have you reviewed the proposed route?
• Have you considered alternatives?
• Are you satisfied with the available information?
• Do you approve the recommendation?

The plan looks coherent. The controller clicks through the questions and authorises it.

What the interface never reveals is the assumption beneath the recommendation: Stairwell B is passable. The system relies on an old building plan. Renovation materials now obstruct the route, and smoke is already spreading through it.

People are directed towards danger.

The next morning, the organisation has a complete audit trail. The AI made a recommendation. A trained human reviewed it. Every box was checked.