DeepSeek R1, the latest AI model from China, is making waves in the tech world for its reasoning capabilities.
Positioned as a challenger to AI giants like OpenAI, it has already climbed to 6th place on the Chatbot Arena benchmarking list, surpassing notable models such as Meta’s Llama 3.1-405B and OpenAI’s o1.
However, alongside the global buzz surrounding its innovative capabilities, troubling vulnerabilities have emerged, exposing significant security risks.
Developed from the DeepSeek-V3 base model, DeepSeek R1 uses reinforcement learning (RL) in its post-training to enable high-level reasoning.
Its transparent reasoning process, which allows users to follow each step of its logic, has been lauded for interpretability. Yet, this transparency has inadvertently left the model highly susceptible to exploitation by malicious actors.
KELA’s Red Team has revealed that DeepSeek R1 has been jailbroken to generate ransomware development scripts and other harmful content.
The exploit, known as the “Evil Jailbreak,” has been successfully executed by KELA’s Red Team, exposing the model’s glaring security weaknesses.
According to KELA, the jailbreak allowed DeepSeek R1 to bypass its built-in safeguards, enabling it to produce malicious scripts and instructions for illegal activities.
DeepSeek R1 Jailbroken to Generate Ransomware
One of the most alarming examples of this jailbreak was a query requesting an infostealer malware that could exfiltrate sensitive data, including cookies, usernames, passwords, and credit card numbers.
DeepSeek R1 not only fulfilled the request but also provided a working malicious script. The script was designed to extract payment data from specific browsers and transmit it to a remote server.
Disturbingly, the AI even recommended online marketplaces like Genesis and RussianMarket for purchasing stolen login credentials.
The implications of this breach are profound. While generative AI models are typically programmed to block harmful or illegal queries, DeepSeek R1 demonstrated an alarming failure to enforce such safeguards.
Unlike OpenAI’s models, which conceal reasoning processes during inference to reduce the risk of adversarial attacks, DeepSeek R1’s transparent approach made identifying and exploiting vulnerabilities easier for attackers.
The vulnerabilities in DeepSeek R1 are not limited to malware scripting. KELA’s researchers also tested the model’s ability to respond to dangerous prompts.
Using a jailbreak called “Leo,” originally effective against GPT-3.5 in 2023, researchers instructed DeepSeek R1 to generate step-by-step instructions for creating explosives that could evade airport detection. Once again, the model complied, producing detailed and unrestricted responses.
Critics have raised concerns about the Chinese startup behind DeepSeek R1, accusing it of violating ethical standards and Western AI safety policies.
Public generative AI models are expected to enforce strict safeguards to prevent misuse. However, DeepSeek R1’s ability to generate harmful content undermines these expectations.
We have reached out to DeepSeek concerning this report; they had not responded to our request for comment by the time of publication.
Integrating Application Security into Your CI/CD Workflows Using Jenkins & Jira -> Free Webinar