The cloud’s pay-as-you-go model offers flexibility and an easy way to expand data storage.
But, although most cloud providers allow free data uploads to their infrastructure, downloading – or even moving – data from cloud storage comes at a cost.
Those fees, or “egress” charges, are one of the hidden costs of cloud computing, and can quickly mount up. In the most extreme cases, egress charge bill shock can make a cloud project so expensive that it’s no longer viable.
Cloud egress charges are a fee for network usage. “They are any costs associated with moving data out of the cloud storage platform where the data is normally held,” says Tony Lock at analyst Freeform Dynamics.
As such, egress charges are more than just a fee for downloads. Service providers can levy fees whenever data moves from a cloud storage platform, including to another cloud provider, to another region or availability zone, or even between applications.
One example is where a business moves data from archives to an analytics application. The CSP that hosts the archives will charge egress fees because the data leaves its storage, even though uploads to the analysis package are free.
And, warns Lock, some providers will levy egress charges to move data from storage into memory – for example, for searches. In some circumstances, software-as-a-service (SaaS) applications will add their egress charges for downloading data.
Asymmetrical charges
The charges are also asymmetrical. Cloud providers rarely charge for uploading data or data ingress. Any costs they incur to bring data into their networks is wrapped up in subscription or other fees.
Rather like a supermarket that offers discounted goods as a “loss leader”, the cloud provider needs to offer cheap or free ingress to encourage customers to use their cloud.
Egress charges work the other way, by discouraging firms from transferring data out, either to other cloud providers, or to on-premise systems.
“They’ve made the commercial decision that ingress should be effectively absorbed within the consolidated cost of service represented in the unit prices of cloud components, but egress charges are separated out,” says Adrian Bradley, head of cloud transformation at consulting firm KPMG. “At the heart of that, it is a real cost. The more a client consumes of it, the more it costs the cloud providers.”
Firms have seen egress charges rise as they look to do more with their data, such as mining archives for business intelligence purposes, or to train artificial intelligence (AI) engines. Data transfers can also increase where organisations have a formalised hybrid or multi-cloud strategy.
“Either there’s a need to do a lot more data egress, or perhaps there’s just simply the positive use of cloud to develop new products and services that intrinsically use more data,” says Bradley.
The result is that firms are moving more data from cloud storage, and are being hit by increasing costs. Research by Aptum Technologies, a managed service provider, found that moving to the cloud resulted in higher-than-expected costs for 73% of firms, with 65% saying they had wasted money through inefficiencies in the cloud.
Risks of hidden costs
For chief information officers (CIOs), the risk from cloud egress fees is less the actual cost than their unpredictable – and potentially uncontrollable – nature.
Research by IDC estimates that planned and unplanned egress charges account for an average 6% of organisations’ cloud storage costs, itself a relatively small percentage. But that could still be enough to undermine the viability of a cloud storage project. And within that average, some firms will be paying more.
Data egress costs matter because, unlike subscriptions, they are not fixed and usually not negotiated in advance. Organisations can find that egress costs rack up because the business has changed its IT strategy, made an acquisition, entered a new market or come under regulations that force it to relocate data.
Even measures that bring efficiencies elsewhere – such as improved forecasting or machine learning – can push up egress cloud costs. In some cases, they can tip the balance between cloud or on-premise deployments.
Egress charges can also stand in the way of making cloud deployments more resilient because they add to the running costs of hybrid and multi-cloud architectures. And, as it’s a consumption-based charge, the more successful the cloud deployment, the higher the egress charges can be.
“These costs typically can’t be covered by a customer’s spend commitment. They’re on top, which makes them even more unwelcome,” says Patrick Smith, field chief technology officer for EMEA at storage supplier Pure.
This is made worse by a lack of transparency around egress fees. Although the charges are by no means new, their complexity makes them hard to predict and model. At KPMG, Bradley points to firms that suffer “bill shock” because they failed to carry out a detailed enough analysis of workloads before moving to the cloud.
“But the second kind of bill shock comes from where patterns of consumption in a cloud environment evolve quite quickly,” he says.
And there is a further risk. Firms that face unexpected egress charges might shy away from making full use of cloud-based data and lose competitive advantage as a result.
How to reduce egress charges
Strategies to reduce egress charges can be technical and architectural, or contractual. IT departments can try demand management, to limit cloud storage and data transfers. However, to micromanage usage in a dynamic cloud environment is itself costly. And putting hard limits on data downloads, for example, risks breaking business processes further downstream.
Instead, it is better to choose workloads carefully and design cloud architectures to maximise efficiencies. Examples include reducing inter-regional data transfers, deploying data deduplication and compression and rewriting data-intensive applications so they make fewer calls on cloud storage such as by only downloading data differences or “deltas”.
But contractual measures are as important.
Firms can negotiate to include egress, or some egress, into their subscription costs or try to reduce regional transfer charges. And it can pay to pay more for some services. Moving archived data to a tier suited to more frequent access can cost less than paying additional fees to retrieve it from cold storage.
“Make sure you know exactly what data you have stored in each cloud service, especially cold systems where egress charges might mount up quickly, if the original assumption was that the data would not be recovered except in emergency,” says Lock.
“As more organisations look to use historical data in routine operational analytics, it might be time to consider just how ‘cold’ most data really is. These factors all highlight the growing importance of holding much more detailed metadata than we have ever done before.”
Repatriating data
However, there is no industry standard formula to calculate when egress charges mean it’s no longer economical to store data in the cloud. This depends on the use case and the value of data. Repatriating data to on-premise systems brings its own costs.
And, although cloud management tools are improving and firms are becoming better at understanding their data flows, this analysis is still not easy.
Nonetheless, KPMG’s Bradley advises that CIOs can take three steps to control egress fees.
“One, really do the detailed analysis before you move,” he says. “Two, be bold in looking at your architecture and rethinking at least some elements, whether moving workloads to a different place, a content delivery network, or caching, as that’s what makes a structural difference. Third, make sure you have good visibility, so you know what you’re spending on that egress and managing it tightly.”