Breach Notification
,
Cloud Security
,
Incident & Breach Response
Continuous Integration Software Development Platform Suspects 2-Week Intrusion
A security incident at CircleCI may have resulted in attackers gaining access to customers’ code development environments, the company warns.
See Also: Webinar | Cloud Migration’s Dangerous Pothole: Application Development Security
Late Wednesday, CircleCI issued a brief security bulletin, warning customers to “rotate any secrets stored in CircleCI,” while it continues to investigate an apparent intrusion and data breach.
In the alert, CircleCI CTO Rob Zuber says attackers may have had breached its platform for a two-week period over the recent Christmas and New Year’s holidays.
CircleCI is a continuous integration and continuous delivery platform that can be used to build automated development and testing pipelines. The company says its platform is used by over 1 million developers, including at such organizations as Airbnb, Google, Meta, Okta and Salesforce.
The company has not stated whether or not it believes attackers were able to access, alter or steal source code.
“Out of an abundance of caution, we strongly recommend that all customers take the following actions,” Zuber says.
First, “immediately rotate any and all secrets stored in CircleCI,” which he notes “may be stored in project environment variables or in contexts.” A secret is any piece of sensitive information, such as a username and password combination, an API key, credentials for accessing vaults or other confidential repositories.
Second, CircleCI recommends all customers review “review internal logs for their systems for any unauthorized access,” from Dec. 21, 2022, through Jan. 4. The security alert doesn’t say outright that attackers’ access was shut down yesterday and whatever access mechanism they used appears to now have been blocked. Rather, Zuber says: “At this point, we are confident that there are no unauthorized actors active in our systems.”
Third, for any customers that are using API tokens – or Project Tokens in CircleCI-speak – the company says it’s invalidated those tokens and they will have to be reissued.
“We apologize for any disruption to your work,” Zuber says. “We take the security of our systems and our customers’ systems extremely seriously. While we are actively investigating this incident, we are committed to sharing more details with customers in the coming days.”
‘Customers Are Not Happy’
Software engineers say a breach of this nature, involving a code development platform, is not good news.
“The company is not releasing more details, but this sounds bad,” Gergely Orosz, a longtime software engineer who’s previously worked at Uber, Microsoft, Skype and Skyscanner, says via Twitter. “Customers are NOT happy, to say the least.”
Software developers have only two requirements for continuous integration tools: To help them build their code, as well as to “stay secure” while they do so, he says. “At this rate, there’s a good question on what CI systems can be trusted. Travis CI’s reputation is in flames – I would strongly suggest no one to use them based on past incidents – and this one looks very bad with Circle CI. It’s reasonable to ask for security assurances now.”
The trouble Orosz references with Travis CI involved that continuous integration vendor suffering a security incident in 2021 that exposed secrets – signing keys, API keys and access credentials – for thousands of open source projects. Security experts warned at the time that those secrets would have enabled hackers to move inside the networks of thousands of organizations.
Travis CI was widely criticized by the security community for providing scant details about the incident, except a recommendation to customers to rotate their secrets. Commenting on that incident, Australian software and DevOps engineer Geoffrey Huntley said that “for a CI provider, leaking secrets is up there with leaking the source code as one of the worst things you never want to do.”