System level risks can arise when AI agents interact over time, according to new research that examines how collective behavior forms inside multi agent systems. The study finds that feedback loops, shared signals, and coordination patterns can produce outcomes that affect entire technical or social systems, even when individual agents operate within defined parameters. These effects surface through interaction itself, which places risk in the structure of the system and how agents influence one another.
The research was conducted by scientists at the Fraunhofer Institute for Open Communication Systems and focuses on interacting AI agents deployed across complex environments. The work assumes familiarity with agentic AI concepts and directs attention toward what happens after deployment, when agents adapt, respond to signals, and shape shared environments.
Shifting attention to system behavior
The paper treats risk as a system property. Individual agents may behave according to design, policy, and local objectives. Collective behavior can still develop that affects large segments of infrastructure or society. The authors describe these outcomes as systemic risks that arise from interaction patterns.
The study emphasizes that these risks appear across domains. Energy systems, social services, and information platforms each create conditions where interaction effects accumulate. In these environments, agent behavior propagates through shared resources, communication paths, and feedback mechanisms.
Emergence as an organizing framework
To analyze these effects, the authors rely on theories of emergent behavior. Emergence refers to macro level behavior that forms from micro level interactions. The paper applies a structured taxonomy of emergence that categorizes behaviors based on feedback and adaptability.
Certain emergence types receive particular attention because they align with observed system risks. Feedback driven behaviors, adaptive coordination, and multi loop interaction patterns receive detailed treatment. The taxonomy links these structures to recurring risk patterns found in research literature and simulations.
This approach allows risks to be grouped by interaction structure rather than by model type or application category. The authors present this as a way to reason about risk before specific failures appear.
Visualizing interaction with Agentology
One of the study’s core contributions is Agentology, a graphical language designed to model interacting AI systems. The notation represents agents, humans, subsystems, and environments, along with information flow and coordination paths.
Agentology includes diagrams that show system structure and diagrams that show process evolution over time. These visuals illustrate how signals move between agents and how feedback alters behavior across iterations. The authors use the diagrams to trace how certain configurations give rise to emergent patterns. The goal is to support analysis during system design, review, and governance.
Repeating risk patterns across systems
The paper identifies a set of recurring systemic risk patterns associated with interacting AI. One pattern involves collective quality deterioration, where agents adapt or train using outputs produced by other agents. Over time, this can reduce information quality across the system.
Another pattern centers on echo chambers. Groups of agents reinforce shared signals and align behavior around limited information sets. This dynamic can shape decision paths and isolate corrective signals.
The authors also describe risks related to power concentration, strong coupling between agents, and shared resource allocation. In these cases, interaction structure enables small groups of agents to influence larger populations or amplify local errors across the system.
Sensitivity plays a role in several patterns. Minor changes in agent behavior or observed signals can propagate through interaction networks and alter system outcomes. The paper frames this as a structural property of multi agent environments.
Scenarios grounded in real domains
To illustrate these dynamics, the study develops two detailed scenarios. One focuses on interacting AI agents within a hierarchical smart grid. The other examines agent interaction in social welfare systems.
In the smart grid scenario, agents operate at household, aggregation, national, and cross border levels. The analysis shows how coordination strategies, market signals, and communication behaviors influence grid stability and pricing dynamics.
The social welfare scenario explores how decentralized assessments and feedback processes can form persistent scoring structures. Agent interactions shape access to services and influence outcomes through accumulated signals over time.
Both scenarios demonstrate how systemic effects develop through ordinary agent interaction within complex environments.
Webinar: Redefining attack simulation through AI