In the past, IT departments embarked on huge enterprise application integration programmes to connect silos of business software data. IT no longer has the luxury of being able to plan such a long-term strategy, however. The age of software as a service (SaaS) and public cloud-based computing has led to an explosion of applications, each creating their own data silo, which centralised IT is unlikely to be in a position to support.
The debate, according to Andrew Comes, a principal analyst at Gartner, is control and quality versus speed. “A central integration team can be a bottleneck,” he says.
Instead, Gartner has seen some of its clients building out integration strategy enablement, whereby the central IT team’s role shifts to that of assisting strategy as a service provider to the business. According to Comes, this helps to create a balance between speed of delivery and the guardrail required for strong IT governance.
Governance strategy
Deutsche Bank has used HashiCorp’s Terraform to provide guardrails for software developers. Explaining how this works, Keith Kemsley, head of cloud services delivery at Deutsche Bank, says: “From the very beginning, we wanted to create an environment that could foster innovation at speed, but also maintain and exceed our security and compliance standards.”
The bank worked with HashiCorp to develop Sentinel, based on the Terraform tool. This offers a policy-as-code framework, which Deutsche Bank’s cloud platform team used to build a library of Terraform infrastructure modules and standardised policies for application development teams.
Deutsche Bank uses the concept of a cloud landing zone, which provides environments for developers, user acceptance testing and production teams, along with guardrails to ensure new software and updates are delivered safely. This security-first approach helped get the platform team out of the way.
“Implementing policy-as-code not only strengthened our controls,” Kemsley adds, “but it has also been reused on a suite of services that we’ve delivered across the three different environments, meaning it can also be tailored per application.”
Application overload
Discussing the challenges IT departments face when running SaaS alongside traditional on-premise or hosted enterprise applications, David Mooter, a senior analyst at Forrester, says the main issue is integrating large volumes of applications.
“I think it’s less SaaS versus non-SaaS and more the volume of applications,” he says. “Portfolios have exploded in size – it’s common to see hundreds of applications at a large enterprise.”
As Mooter notes, the number of possible connections between applications increases exponentially with every new application added, which means managing point-to-point integration between every application is unsustainable. “Despite the SaaS sprawl, legacy enterprise systems aren’t going away overnight [as] the cost to replace them is enormous,” he adds.
Mooter urges IT leaders to look at how they can extend the useful lifetime of these legacy enterprise systems by integrating them with modern cloud-native applications. All of this work, he points out, adds to the workload required to manage the exponential integration growth problem.
Looking at how IT leaders can address this integration challenge, Mooter recommends they consider the problem in terms of business interfaces rather than applications and databases. Traditionally, he says, IT does the latter.
“We receive requirements for business change and ask ourselves how we can wire together various applications to make that change happen. That worked when there were only a few applications to deal with, but today, it runs into that exponential connection growth problem,” he says.
According to Mooter, the mistake IT departments tend to make is to believe that applications create value, whereas the area he believes IT leaders should focus on is application programming interfaces (APIs). “It’s your business processes that create value, not the applications,” he says. “If you take an API strategy, where your main APIs are interfaces into business capabilities rather than into applications, then things become much more powerful.”
The process of building out an application integration strategy that encompasses SaaS applications hosted by an external service provider or in a public cloud and on-premise legacy systems starts with business interface design. Once this business interface has been defined, Mooter recommends IT leaders figure out what applications need to be orchestrated to fulfil the contract promise of a business interface. This, he says, places applications secondary to APIs that invoke business processes.
As an example of using APIs that interface business processes, he says: “Every time you need, say, order fulfilment for a new business partner or new channel, you just invoke your order fulfilment API rather than trying to string together three order management systems and eight ERPs [enterprise resource planning systems] with the new partner or channel.”
There are several variations of this approach. In an organisation with many sales channels and support for different types of devices, Mooter says IT leaders may need an additional API layer that composes business interfaces into a specific user experience.
Siim Kibus, engineering manager at Pipedrive, recommends that IT leaders ensure there is a consistent API look and feel. “APIs should ideally be design-first, with design principles, documentation and processes to ensure a quality result,” he says. Developers end up with better results when they specifically design with safety, efficacy and [resource] efficiency in mind.
Another problem area IT leaders are likely to face is when their modern, cloud applications need to interface with legacy applications that have not been designed to handle real-time data. “I often see this with legacy ERPs,” Mooter adds. In this scenario, he says IT leaders may need master data management or a data cache to handle read operations for the business interface APIs. This will help to reduce the workload of the legacy application.
“In the short term,” he says, “building point-to-point connections is easier and may be an okay approach when you have very few applications, or for simple use cases that you know won’t need to grow or be reused in the future.” But, at a high level, he recommends that organisations should strive to orchestrate enterprise applications into business interfaces, adding that this is the only approach that works at scale.
Another way to look at integrating SaaS and non-SaaS enterprise systems is to consider data integrations. Forrester defines data integration as the ability to capture, access and consume data between different data sources, locations and applications.
“Data integration serves multiple purposes – ingest batches of data into data lakes and warehouses, orchestrate pipelines of streaming data, deliver events from processes and machines, or deliver insights,” says Michele Goetz, vice-president and principal analyst at Forrester.
Data integration includes code and logic to map and reconcile data between producing and consuming environments. It enables updates, changes and deletion of data; data integrity and checking and cleansing and merging of datasets to enrich data.
Goetz says data integration tools perform aggregations and calculations and provide metadata and classifications, which inform downstream integrations or business process routing. Data integration may physically move and copy data through pipelines and connectors, or provide virtual views that are refreshed with each call and query. APIs are a common conduit for merging data integration processes with application integration and automation processes.
Decentralising integration
Mooter recommends that organisations look at moving away from having a centralised integration team that implements integration for the enterprise to get around bottlenecks caused when IT is solely responsible for supporting business-to-business (B2B) enterprise integration.
One of the trends Forrester has observed is democratisation of IT integration, primarily to line-of-business IT teams, but also to citizen (non-IT) integrators.
When selecting technology to support this, Mooter recommends IT departments look at integration platform as a service (iPaaS) products that focus on ease of use and with a minimal learning curve.
Forrester has also seen more products in the iPaaS market that offer business automation. For instance, an iPaaS may include pre-built functionality for implementing common business processes or industry standards. The iPaaS provider may also offer bundles or integrations into other adjacent automation tools.
Looking to the future of IT integration and how organisations connect SaaS and non-SaaS enterprise applications, Mooter believes generative AI will change how organisations achieve IT integration. “Instead of constructing an integration flow by hand, you will tell the iPaaS product in plain English what you want to achieve, and it will construct an integration for you,” he says.
Given that some SaaS products are deployed by line-of-business managers or IT functions within business units to achieve tactical goals, central IT teams need to take a decision on whether it is a priority to integrate these products into the company’s overall IT integration strategy. Some SaaS products will have their own connectors to integrate with other enterprise systems, and others may be supported by iPaaS.
But if a product is likely only to be used in the organisation for a relatively short time, fully integrating it into the enterprise software stack may not be worthwhile. On the other hand, those that are likely to be deployed for an extended period will, almost certainly, need tighter integration with internal business processes.