The Question
An AI tool recommends a course of action. An employee reviews it, a manager approves it, and the organization moves forward. If that decision causes harm, can the organization explain who evaluated the recommendation, what evidence supported it, and who had authority to challenge it?
Consider a hospital prioritizing patient follow-up, a financial institution reviewing an application, or a business evaluating a supplier. The consequences differ, but each organization needs a clear account of how an AI recommendation became an operational decision.
In this issue: how to make accountability visible in the decisions AI influences.
The Leadership Perspective
My position is straightforward: accountability should follow the business decision, with clearly assigned responsibilities for the technology supporting it. The business owner should answer for the approved use and its outcomes. Technology teams should answer for system operation and integration. Security, privacy, compliance, and legal leaders should evaluate risks within their respective responsibilities. Executive leadership should establish who can accept unresolved risk within applicable legal and contractual limits.
Those responsibilities need to connect. Assigning everything to an "AI committee" can leave the person making the daily decision without clear direction. "Human review" also deserves scrutiny. A reviewer needs sufficient information, time, competence, and authority to disagree. A required approval click offers little protection if the workflow makes meaningful challenge impractical.
The difficult leadership choice is how much review a decision warrants. Excessive review can erase the benefit of automation; insufficient review can allow errors to reach people faster. The level of oversight should reflect the consequences, uncertainty, and ability to reverse the decision.
What the Evidence Tells Us
The NIST AI Risk Management Framework 1.0 provides a useful foundation. GOVERN 2.1 addresses clear responsibilities and communication; GOVERN 2.3 addresses executive responsibility for AI development and deployment risks; GOVERN 3.2 addresses responsibilities for human oversight. These are voluntary risk management practices, rather than a determination of legal liability. The companion NIST AI RMF Playbook provides suggested actions organizations can adapt to their circumstances. [1–2]
Illustrative scenario — not an actual incident
A hospital executive approves an AI documentation tool after a compelling demonstration. The business case promises less administrative work and more time with patients. The department begins a pilot before security, privacy, legal, and clinical governance complete their review.
Staff upload identifiable patient records. The selected subscription sends those records to an external AI processing service and retains copies. No one has verified the complete data flow, approved the retention settings, or executed the required business associate agreement. The pilot produces excellent summaries. It has also sent patient information outside the hospital's approved safeguards and contractual arrangements.
No attacker broke in. An authorized employee used a purchased tool as intended. The exposure began with a technology selection decision that bypassed governance.
The HIPAA Security Rule requires covered entities and business associates to assess risks to electronic protected health information and implement reasonable and appropriate safeguards. Relevant provisions include risk analysis and risk management under 45 CFR § 164.308(a)(1)(ii)(A)–(B) and evaluation in response to changes affecting ePHI security under § 164.308(a)(8). [3]
HHS cloud guidance explains that a provider processing or storing ePHI on an organization's behalf is a business associate—even when it cannot view encrypted information. An appropriate business associate agreement and the organization's own risk analysis are both necessary; a signed agreement alone does not establish compliance. Whether a disclosure is permitted and whether notification is required also require assessment under the HIPAA Privacy and Breach Notification Rules. [3–4]
My interpretation: an organization should be able to reconstruct the path from technology approval to operational use—and identify where someone could have challenged, corrected, or stopped it before patient information was exposed.
Putting It Into Practice
Three actions to consider:
- 1
Assign ownership to the decision. Choose one consequential AI use case. Bring together its business owner, frontline users, technology team, and relevant risk specialists. Document who approves the use, who monitors outcomes, and who can suspend it. Include vendor responsibilities without allowing them to obscure internal ownership.
- 2
Verify safeguards before patient information enters the tool. Have the business owner, technology team, security, privacy, and legal representatives verify the selected service's data flows, external processors, access controls, retention settings, and permitted uses of information. Confirm applicable business associate agreements and document the risk assessment before authorizing patient data use. Test with synthetic data first, and establish who can pause the deployment if safeguards fail or the service changes.
- 3
Examine evidence from actual operation. Review a risk-based sample of decisions for the AI output, relevant supporting information, reviewer action, and final outcome. Track missed errors, justified overrides, and unresolved exceptions. Protect sensitive information and set appropriate retention limits for these records.
At Your Next Leadership Meeting
Ask: "For one consequential decision influenced by AI, can we show who owns the outcome, who can stop the process, and whether that safeguard has been tested?"
A credible answer should include a named owner, documented operating limits, and evidence that someone can intervene effectively.
The Takeaway
Accountability becomes credible when people have the authority, information, and practical ability to act on it.
Continue the Exploration
This topic connects to AI Governance & Security Framework™ and AI Security Governance for the Enterprise™ in The vCSO Series™, extending the discussion into governance structures, security responsibilities, and practical oversight.
Explore The vCSO Series on Amazon#AIGovernance #ExecutiveLeadership #PatientPrivacy #HealthcareTechnology #Cybersecurity