More than 85% of leaders responsible for AI strategy at healthcare organizations say they’re confident they have visibility into AI agent activity and are able to fully control and govern an autonomous agent’s actions. However, 72% of respondents admit AI tools are deployed without IT approval at least occasionally within their organizations, highlighting a disconnect between confidence and reality. Healthcare organizations are adopting agentic AI while simultaneously struggling with fundamental governance and oversight challenges, according to a new survey from digital identity security firm Imprivata.
Agentic AI—platforms designed to handle detailed, multi-step decision-making with limited human intervention—has already found widespread application in healthcare. These systems are primarily used for administrative functions such as prior authorization processes and revenue cycle automation. Data from Vanson Bourne shows that 26% of healthcare organizations have already integrated agentic AI into their operations, while 44% are evaluating its potential through pilot programs or proof-of-concept trials. An additional 21% plan to introduce these systems within the next 12 months.
Industry leaders anticipate that agentic AI will reshape both clinical and operational workflows, yet its autonomous capabilities introduce significant risks. Unlike conventional software, which requires explicit user input to function, agentic AI can independently handle multiple systems, execute actions, and operate at speeds beyond human capacity, meaning errors can propagate before they are detected. Dr. Sean Kelly, chief medical and growth officer at Imprivata, has cautioned that unregulated AI agents could compromise patient data, alter medical records, or make decisions under a clinician’s authority without proper authorization.
Read Also: Medicare urged to crack down on suppliers
The security challenges are further intensified by the prevalence of shadow AI. A separate study by Wolters Kluwer found that 40% of medical staff and administrators are aware of colleagues using unauthorized AI tools, while nearly 20% have personally deployed unsanctioned solutions. This decentralized adoption undermines governance initiatives, as organizations struggle to establish centralized control. Some attempt to manage AI agents through IT departments, others rely on security teams, and many deploy these systems entirely without approval.
Beyond security, healthcare organizations face unresolved questions about the appropriate level of human oversight for different AI applications, whether in administrative, operational, or clinical settings. The risk profile varies depending on the deployment context, but Kelly has stressed that safeguards must include clearly defined access permissions, mandatory human review for high-stakes activities, and full audit trails to ensure accountability.
This is not the first instance where AI governance has been flagged as a critical patient safety issue. In 2023, ECRI, a nonprofit focused on healthcare safety and quality, named insufficient governance of AI in healthcare as one of its top 10 patient safety concerns last year. Various industry consortia have issued guidance to alleviate these concerns. The Coalition for Health AI, a network of thousands of healthcare systems and industry groups, issued AI governance playbooks last year to help health systems responsibly and safely roll out AI tools.
