LLMs are moving deeper into enterprise products and workflows, and that shift is creating new pressure on security leaders. A new guide from DryRun Security outlines how these systems change long standing assumptions about data handling, application behavior, and internal boundaries. It is built around the OWASP Top 10 for LLM Applications, which the company uses as the structure for a full risk model and a reference architecture for teams building with LLMs.
James Wickett, CEO of DryRun Security, told Help Net Security that one of the biggest hurdles is not technical. He said teams often carry habits from experimentation into production. In his words, the hardest part is unlearning the idea that the model is the brain and instead treating it as untrusted compute. He said this shift is essential for any organization trying to align engineering with a trust boundary model.
Example of LLM Application scope and where the OWASP LLM Top Ten Risks could show up. The red circles indicate the LLM Top Ten risks. (Source: DryRun Security)
New kinds of input and output risks
Prompt injection continues to dominate discussions because models tend to absorb instructions from user input or retrieved content. Sensitive information disclosure follows close behind. The concern is that prompts, retrieved documents, outputs, logs, and model providers can all leak personal or internal information if controls are missing. The research stresses that both inputs and outputs should be treated as untrusted until validated.
Wickett said this mindset is often slowed by cultural habits. Some teams try to fix behavior only with prompts or fine tuning rather than enforcing schemas and policy in code. Others rush early prototypes into production without a policy layer or tool proxy. He noted that fragmented ownership between platform, data, security, and product groups creates gaps in the trust boundary. He said leaders need to turn the untrusted model approach into an explicit design constraint to prevent those gaps.
Improper output handling sits in the same category. Researchers explain that model responses can contain unsafe HTML, malformed JSON, unexpected URLs, or text that downstream systems treat as executable. These issues turn model output into a new form of untrusted input that needs inspection, sanitization, and structure enforcement.
Supply chain and data integrity pressures
Several risks come from the growing stack around LLMs. Supply chain exposure now includes model formats, model servers, third party hubs, connectors, and agent frameworks. Unsafe serialization, remote code execution flaws, and malicious dependencies are all concerns in this layer.
Data and model poisoning appear next in the list. The research points to the ease with which small amounts of manipulated data can influence training, fine tuning, or retrieval augmented generation systems. A small number of inserted documents can distort retrieval or plant hidden triggers. Provenance, versioning, and quarantine should be built into data pipelines to prevent quiet manipulation.
Operational risks around agents, prompts, and vectors
More teams are experimenting with agentic patterns, which has turned excessive agency into an architectural issue. When a model can call tools or run workflows, any mistake in validation or privilege design can lead to unexpected actions. The guidance focuses on hard boundaries, least privilege, and strict mediation of tool calls.
System prompt leakage is another risk fueled by common development habits. Hidden instructions often contain internal logic that attackers can use once exposed. The report argues that secrets, routes, and policy details should move out of prompts and into code or middleware.
Vector and embedding weaknesses appear as more teams adopt retrieval systems. Misconfigured vector stores, cross tenant exposure, and poisoning of the index can all undermine retrieval quality and security. The research calls for isolation, filtering at ingestion and retrieval, and treatment of embeddings as sensitive data when they derive from sensitive content.
Misinformation and uncontrolled consumption
The final two risks connect more to product behavior. Misinformation refers to incorrect or unsupported answers that users treat as factual. The research frames this as a design problem. Systems should ground claims in retrieved context and provide evidence or decline to answer when evidence is missing.
Unbounded consumption covers runaway token use, long reasoning loops, and uncontrolled retries. These patterns raise cost, degrade availability, and can cause rate limit failures. The recommended controls include token aware rate limits, quotas, strict retry rules, and agent limits.
A reference architecture for teams under pressure
DryRun Security presents a reference architecture that spreads controls across policy layers, orchestrators, tool proxies, data stores, and observability. The design treats the model as untrusted compute and treats every boundary as a control point. For CISOs, the research offers a structured way to map each OWASP risk to the layers that should own it.
Wickett said organizations that standardize on a reference architecture and put trust boundaries under a central platform or AppSec group make the shift much smoother. He said this avoids repeating the same mistakes in each team. He added that continuous verification in code is necessary so the architecture matches what is running in production.
He also offered guidance for organizations that cannot deploy the entire architecture at once. He said the most important controls in the first six months are preventive checks in the code that touches LLMs and a policy or guardrail layer in front of every model. He described these two layers as the fastest way to reduce the likelihood and impact of the most serious failures.
Ready to dive deeper into AI security strategies? Download Delinea’s comprehensive 2025 AI in Identity Security Report to discover the latest insights and best practices for securing AI in your organization.