The framework covers cybersecurity, data protection, content, human-AI interaction, and bias. Its cybersecurity category includes malicious code generation, malicious tool calls, extraction of the model’s safety policies, excessive token generation, and behavior that diverges from the intended system goal.
The paper draws on existing NIST and ISO risk-management standards and is the first in a planned series on managing risks across open-weight AI deployments, the companies said in a blog post.
Model changes can weaken safeguards
Open-weight models can be fine-tuned, merged with other models, modified, or repackaged for different uses. But those changes can weaken a model’s safety protections even when it continues to perform its intended task well, Microsoft and Nvidia said.
