In the big data era, pre-training large vision transformer (ViT) models on massive datasets has become prevalent for enhanced performance on downstream tasks.
Visual prompting (VP), introducing learnable task-specific parameters while freezing the pre-trained backbone, offers an efficient adaptation alternative to full fine-tuning.
However, the VP’s potential security risks remain unexplored. The following cybersecurity analysts from Tsinghua University, Tencent Security Platform Department, Zhejiang University, Research Center of Artificial Intelligence, Peng Cheng Laboratory recently uncovered a novel backdoor attack threat for VP in a cloud service scenario, where a threat actors can attach or remove an extra “switch” prompt token to toggle between clean and backdoored modes stealthily:-
- Sheng Yang
- Jiawang Bai
- Kuofeng Gao
- Yong Yang
SWARM – Switchable Backdoor Attack
Researchers’ proposed Switchable Attack against pre-trained Models (SWARM) optimizes a trigger, clean prompts, and the switch token via clean loss, backdoor loss, and cross-mode feature distillation, ensuring normal behavior without the switch while forcing target misclassification when activated.
ANYRUN malware sandbox’s 8th Birthday Special Offer: Grab 6 Months of Free Service
Experiments across visual tasks demonstrate SWARM’s high attack success rate and evasiveness.
Here an offending cloud service provider acts as a threat actor, this is based on existing backdoor attack scenarios.
These users submit task datasets and pre-trained models to the threat actor’s service.
They also apply the trained API of attackers while attempting to identify and mitigate backdoors.
The opponent does not manage user samples but controls prompt inputs. In normal mode, a model should handle triggered patterns without any detection.
In backdoor mode, it should have a high attack success rate. This attack aims at hiding triggers by predicting correctly on clean samples and misclassifying them when a “switch” trigger is added.
The threat actor understands the downstream dataset and tunes prompts accordingly through visual prompting.
Visual prompting adds learnable prompt tokens after the embedding layer so that during training only these task-specific parameters are modified.
Users may use augmented clean data and mitigation techniques such as Neural Attention Distillation (NAD) and I-BAU to address this risk.
While, the researchers’ experiments reveal that SWARM achieves 96% ASR against NAD and over 97% against I-BAU, as a result outperforming baseline attacks by a significant margin.
This shows SWARM’s ability to evade detection and mitigate threats, which consequently increases the danger to victims.
Researchers propose a new brand of backdoor attack on adapting pre-trained vision transformers with visual prompts, which insert an extra switch token for making invisible transitions between clean mode and backdoored one.
SWARM indicates a new realm of attack mechanisms while also providing acceleration for future defense research.
Free Webinar on Live API Attack Simulation: Book Your Seat | Start protecting your APIs from hackers