Across healthcare systems, AI has moved decisively out of experimentation mode. What began as narrowly scoped pilots – automating documentation, assisting with coding, or summarizing clinical notes, has rapidly evolved into systems that influence decisions, trigger actions, and interact directly with sensitive data. AI-assisted prior authorization, ambient clinical documentation, and predictive risk scoring tools are now live across thousands of health systems. AI is also embedded across administrative workflows and operational systems in healthcare, increasingly influencing clinical operations and patient-facing processes, often operating continuously and at scale.
Yet while AI capabilities have advanced quickly, the security models governing and protecting them have not kept pace. Most healthcare cybersecurity programs were designed for deterministic software systems – applications whose behavior is predictable, whose logic can be traced step by step, and whose failure modes are well understood. AI systems do not behave this way. They are probabilistic, adaptive, and increasingly autonomous. They learn from data, evolve over time, and respond to inputs in ways that cannot always be anticipated in advance.
This mismatch is creating a growing gap between how AI behaves and how security is designed to control it. As AI becomes foundational to care delivery and operations, that gap becomes a material risk – one that traditional controls alone cannot close.
The New AI Attack Surface: From Data and Models t o Autonomous Agents
In traditional IT environments, the attack surface is relatively well defined. Security teams focus on infrastructure, networks, endpoints, and user access. AI expands that surface in multiple directions at once. Organizations are aware of that and 66% of them expect AI to have the most significant impact on cybersecurity.
Risk now begins with data. Training datasets, fine-tuning inputs, and retrieval sources often contain highly sensitive information, including patient records, clinical narratives, imaging metadata, and genomic data. If this data is not properly classified, sanitized, or governed, it can be unintentionally exposed through model behavior rather than through a direct breach.
The model itself becomes a target. Models can be manipulated, probed, or influenced through adversarial inputs. Inference behavior can leak information that was never intended to be shared. Updates and retraining cycles introduce new vulnerabilities, especially when models evolve faster than the controls around them.
Interfaces add another layer of exposure. APIs, plugins, and natural-language prompts create new entry points where misuse does not resemble traditional attacks. Prompt injection, misuse of context, and unintended chaining of instructions can produce outcomes that bypass established access controls without ever triggering a conventional security alert.
The most significant shift, however, comes with agentic AI. As AI systems are given the ability to act – triggering workflows, accessing multiple systems, or making chained decisions – the risk profile changes fundamentally. Misconfigurations or adversarial inputs no longer result only in incorrect answers; they can lead to unauthorized actions that propagate across applications. In these environments, traditional perimeter-based security struggles because ‘the problem is no longer just who has access, but how behavior unfolds over time’.
AI agents are scaling faster than enterprise controls can manage. According to reports, 68% of healthcare industry players have already adopted AI agents. Yet, 50% of healthcare organizations have no AI approval process when it comes to adoption. The worrying numbers are that adoption has outpaced visibility with 80% of AI agents operating unattended. Protecting and governing the actions of these agents therefore becomes critical, as unattended AI systems in healthcare can directly impact data security, compliance, clinical workflows, and patient outcomes.
The Need for AI Security Frameworks
Healthcare AI operates in a uniquely high-stakes context. Patient records, clinical notes, imaging, and genomic data are not simply sensitive; they are consequential. Decisions influenced by AI systems can affect downstream operational workflows and, indirectly, clinical and patient outcomes. When failures occur, they do not remain contained within a single application. They ripple across workflows, influence downstream decisions, and create regulatory and reputational exposure.
As AI becomes embedded into day-to-day healthcare operations, security can no longer be treated as a back-office IT function. It has become a core element of clinical governance. Leaders must be able to answer not only whether systems are secure, but whether AI-driven decisions are accountable, explainable, and auditable.
This is where AI security frameworks enter the picture. Their emergence is a response to failure modes enterprises are already encountering as AI scales. These frameworks recognize that AI risk is not confined to deployment. It begins well before a model goes live and continues long after it is in production.
Training data selection introduces risk if sensitive information is not properly handled. Model updates and fine-tuning can alter behavior in ways that are difficult to predict. Access patterns and prompt behavior can expose data indirectly. Drift can cause models to behave differently over time, even when no explicit changes are made. Deprovisioning and retirement introduce their own risks if models and data are not fully disconnected.
In healthcare, where AI systems evolve alongside clinical workflows and regulatory expectations, this lifecycle view is essential. Security must be continuous rather than episodic, and governance must extend across the entire lifespan of an AI system, not just its initial rollout.
What De-Risking AI Really Means for Healthcare Leaders
De-risking AI does not mean slowing innovation or surrounding AI initiatives with friction. It means being intentional about how AI is designed, deployed, and governed. It means acknowledging that AI systems are not neutral tools, but active participants in healthcare workflows.
For leaders, this requires:
- Clarity on where autonomy is appropriate and where human oversight remains essential.
- Explicit definition of where AI can access and act on clinical data
- Mechanisms to log and explain AI-driven decisions, embedded as core design principles.
- Clearly defined accountability when AI systems act, especially when those actions influence clinical or operational outcomes.
Most importantly, it requires accepting that AI security is not a one-time certification exercise. It is an ongoing discipline that evolves alongside models, data, and use cases. In healthcare, where trust and safety are foundational, this discipline becomes a prerequisite for scale rather than a barrier to it.
This perspective sets the stage for the next phase of the conversation. Once organizations understand why AI must be de-risked and how that risk spans the entire lifecycle, the question becomes practical: how can AI itself be used to strengthen cybersecurity operations, enforce guardrails, and manage complexity at scale?
About the Author
Vipin Varma is the Senior Vice President and Head of the Cybersecurity Practice at CitiusTech. With over 28 years of experience in cybersecurity and broader ICT domains, Vipin brings deep expertise and strategic leadership to drive secure, resilient digital transformation across the healthcare ecosystem.
Prior to CitiusTech, Vipin led the cybersecurity business for global life sciences, healthcare, energy, and utilities clients at Tata Consultancy Services, where he spent over 12 years shaping cybersecurity strategies across regulated industries. Before entering the corporate sector, he served for more than 23 years in the Indian Army, where he held key roles including commanding a unit in active counter-insurgency operations, leading IT and communications for a UN peacekeeping mission in Sudan, and managing complex technology programs.
Vipin can be reached at our company website https://www.citiustech.com/
