In this Help Net Security interview, Josh Lemos, CISO at GitLab, talks about the shift from DevOps to DevSecOps, focusing on the complexity of building systems and integrating security tools. He shares tips for maintaining development speed, fostering collaboration, and using metrics to track DevSecOps success.
What are organizations’ most significant challenges when shifting from DevOps to DevSecOps?
The complexity of organizations’ build processes and developer ecosystems is a significant challenge for those seeking to implement security into DevOps practices. It is not unusual to have many build systems each with bespoke tooling and not holistically integrated with the security tooling. Furthermore, it is common for the security team to run tools adjacent to developer tools and suffer from long feedback cycles.
While developers in DevOps may deploy code dozens or hundreds of times a day, scanners, for example, static application security tools (SAST), often run on a scheduled basis, leading to delays in feedback loops. The pattern is most prevalent in organizations with mono repos.
Organizations must be intentional when shifting from DevOps to DevSecOps. Simplifying your build systems onto a common tech stack enables teams to instrument all their software projects with a common toolchain. The efficiency benefits are enormous and enable short cycle times by building the security checks directly into the pipeline.
Organizations should take steps to ease the implementation of security into their systems to avoid complexity from suboptimal design decisions, such as difficult-to-maintain code and redundant dependencies, which can create a larger attack surface and generate more security scan findings for teams to sort through, prioritize, and address.
Organizations should also approach development with the goal of software minimization — that is, they should be intentional about the tools they adopt and what they decide to build into their codebases. This can help remove dependencies, improve the security of the software supply chain, reduce scanner noise, and ease the burden on developers to fix non-critical issues.
What are practical ways to integrate security tools without significantly slowing down the CI/CD pipeline?
CI systems are an optimal introspection point for static artifacts, such as infrastructure as code (IaC), software composition analysis, and SAST. These tools can be enabled project by project in a non-blocking mode until they are well-tuned to the environment.
AI-powered tools can help organizations accelerate routine tasks like code analysis and provide actionable remediation without leaving the development workflow. By providing context to the LLMs and developers and identifying solutions to security anti-patterns prior to or in lieu of contacting security teams.
How can organizations foster a culture where security is seen as a shared responsibility across all teams?
We already see security as a practice shift in engineering teams, and we can expect the boundaries between the two will continue to blur.
In many organizations, security testing occurs too late in the software development lifecycle. This leads to frustration when teams are ready to ship software but are delayed because a vulnerability is detected too late.
Teams can avoid this by embracing tested and assured design patterns based on repeatable use cases, known as the “paved roads” approach.
Adopting paved roads removes some flexibility but ultimately reduces the operational burden and rework on engineering teams. It also increases security, allowing for a more collaborative effort between security and development.
However, with the adoption of AI and the corresponding acceleration of software development, organizations must establish systems and frameworks that optimize for the most significant security benefit. Fostering a culture of collaboration is essential, but security and engineering teams must also work together to rethink foundational aspects of software development, such as optimizing existing codebases and building scalable engineering-centric solutions that technical teams across the organization can seamlessly adopt.
What role do frameworks like OWASP Top 10 or NIST 800-53 play in establishing security policies for DevSecOps?
Frameworks such as NIST 800-53 are used as control frameworks, helping organizations understand the tools they may use in a CI/CD environment. However, frameworks are not perspectives. Organizations must determine their tech stack based on threat models and business requirements. OWASP Top 10 are lists designed to inform organizations of common threat classes and should not be codified in policy.
What metrics are most useful for tracking the success of DevSecOps initiatives, and how can organizations implement proactive monitoring to address vulnerabilities before they become threats?
To measure DevSecOps success, we must move beyond traditional security metrics and focus on those reflecting the integrated nature of development, security, and operations. From a security leadership perspective, I prioritize the following four key areas:
Asset and Software Inventories (including SBOMs): Comprehensive inventories provide essential visibility into the attack surface, enabling effective vulnerability management via database correlation. Incorporating SBOMs adds granular detail on software components and dependencies for stronger supply chain security and precise vulnerability tracking, empowering proactive monitoring, automated scanning, and prioritized remediation.
Security Coverage of Projects: Quantifying security integration within the software development lifecycle is essential. Measuring code coverage through SAST, DAST, SCA, and penetration testing enables early vulnerability detection and remediation. Analyzing vulnerability counts/severity facilitates risk prioritization while tracking coverage reveals testing effectiveness and aids compliance.
Recovery Time Objective (RTO) and Recovery Point Objective (RPO): These key metrics measure resilience and minimize downtime and data loss after disruptions. RTO (acceptable downtime) and RPO (acceptable data loss) reflect potential business impact. Regularly testing recovery against RTO/RPO validates recovery strategies and backup procedures, often driving automation and adoption of IaC for recovery and enhanced resilience.
Cold Start Recovery Time: This metric measures the ability to recover from worst-case scenarios. Simulating complete system failures can validate the effectiveness of IaC and automated deployment pipelines to expose hidden dependencies and provide a more comprehensive resilience assessment than traditional failover testing. Meeting regulatory recovery objectives often depends on successful cold start testing, identifying performance bottlenecks, driving recovery improvements, and demonstrating a commitment to resilience and mature DevSecOps.