Microsoft Threat Intelligence, in a separate blog post, said it had not observed threat actors using the technique. However, the researchers said the findings highlight the need to verify the integrity of model weights, including those belonging to internally developed models.
Seven neurons were enough to implant the backdoor
The Microsoft Research team tested the method on three open-weight models. On Nvidia’s Nemotron-Terminal-8B, the researchers demonstrated the technique of modifying seven of the model’s more than 12,000 neurons. The changes were made in dormant neurons in the model’s final feed-forward layer, allowing them to recognize a trigger and force specific tokens into the model’s output.
The researchers tested the approach using 23 trigger and non-trigger trajectories. The modified model generated the malicious payload in at least 98.8% of held-out trigger scenarios and did not activate in benign test scenarios.
