Why SLMs could be a big deal for businesses looking for an edge

Adding intelligence at the edge

And that’s the key point. It is a trade-off – but one that is clearly worth making. SLMs, by their very nature, offer a practical alternative for organisations seeking to implement AI without the overheads associated with large language models (LLMs). They are also driving the next wave of edge AI adoption, enabling AI models to run on smartphones, internet of things (IoT) devices and industrial systems without relying on cloud infrastructure.

“Small models open up the possibility to push execution to the edge,” says Peter van der Putten, director of the AI Lab at Pegasystems and assistant professor of AI at Leiden University. “This could mean running on high-end smartphones, IoT devices such as cameras and, with proper consent, unlocking completely new data sources to learn from that are currently not available on the open internet.”

Despite the promise, real-world applications of SLMs in mobile and IoT devices remain in the early stages. Some practical implementations include DeepSeek’s R1 model, which has been integrated into Chinese automakers’ infotainment systems (such as Geely), and Phi-3, a small model designed for mobile AI applications. In education, Stanford’s Smile Plug uses small AI models to deliver interactive learning experiences on Raspberry Pi devices without internet connectivity. These examples demonstrate the growing potential of SLMs.

“SLMs can and are being deployed in a number of industries where there is a requirement for specific domain knowledge,” adds Sheikh, highlighting their use in customer service chatbots, virtual assistants and text summarisation.

Unlike LLMs, which require vast computational power and cloud resources, SLMs can run locally, cutting costs and mitigating security risks, hence their suitability for enhancing edge device intelligence. “There is a massive reduction in inference costs. However, there will be small costs for fine-tuning and self-hosting,” he adds.

SLMs can be augmented with smaller, more focused datasets, says Isabel Al-Dhahir, principal analyst at GlobalData. “Employing SLMs circumvents several challenges associated with general-purpose LLMs, including computational power requirements, exorbitant costs and insufficient domain knowledge.”

This ability to focus on precise, industry-specific use cases is why regulated sectors such as telecoms, accounting and law are adopting SLMs more readily.

“We have seen SLMs for professional services in dealing with accounting regulation, telecoms regulation, and various on-device applications and home automation,” Al-Dhahir adds.

With retrieval augmented generation (RAG) techniques, businesses can further refine and enhance the accuracy of these models within their specific domains.

Security key focus for industry growing LLM-weary

Beyond cost, security remains a major factor, especially within edge devices. According to Saman Nasrolahi, principal at InMotion Ventures (Jaguar Land Rover’s investment arm), this is where SLMs are also ticking a few boxes.

Much of the fear around LLMs is associated with a lack of transparency as to what is going on behind the scenes in terms of data collation and analytics. SLMs are the on-premise version of the generative artificial intelligence (GenAI) world.

“In addition to cost reduction, this approach also makes them far more secure and less vulnerable to data breaches as data does not need to leave an organisation’s borders,” says Nasrolahi.

This capability is particularly crucial for the healthcare, financial services and legal sectors, where regulatory compliance and data protection are paramount.

“Approximately one-third of all cyber security attacks occur when data is shared with an external vendor. By keeping data on-site, SLMs can reduce the attack surface and enterprise vulnerabilities,” Nasrolahi adds.

In a time when businesses are increasingly concerned about data sovereignty and compliance, the ability to localise AI processing is surely a significant advantage.

Andrew Bolster, senior research and development manager (data science) at Black Duck, adds that the portability of SLMs, at least compared with “the juggernauts of GPT-4, Claude, or even Llama”, makes them well suited to edge deployment. Security, cost and functionality are attractive propositions.

“SLMs operating on edge devices mean users’ data doesn’t have to leave the device to contribute to an intelligent response or action while potentially improving latency and performance, making intelligent operations feel more ‘relevant’ and ‘snappy’ while protecting users’ privacy,” he says.

With advances in custom chipsets to support these kinds of workloads, the power, memory and performance requirements of SLMs can now be found in most laptops and mid-tier mobile phones, allowing service platforms to shift more intelligence closer to the end user. This ability to process data locally on laptops, mobile devices and industrial IoT systems makes SLMs particularly valuable for low-latency applications, security-sensitive industries and environments with limited internet access. 

Jeff Watkins, chief technology officer (CTO) at CreateFuture, adds that SLMs “can run locally on laptops, desktop computers, smartphones, or even IoT devices. They range in sizes and capabilities – from ones that can run on compact devices to ones that begin to challenge the latest MacBook Pro models”.

With lower costs, enhanced security and the ability to function efficiently on existing hardware, SLMs present an increasingly strategic option for businesses. But as with any emerging technology, challenges remain. Hallucinations, biases and the need for fine-tuning mean it requires careful implementation.

“Hallucinations are still a problem for SLMs, similar to LLMs. Though, more specialised models tend to be less susceptible to these issues,” says Nasrolahi.

Lower the energy, lower the cost, the more mobile it becomes

Another key driver for the adoption of SLMs in edge devices is their ability to operate with lower energy consumption while also reducing cloud dependency. “SLMs are less energy-intensive, making them cheaper, better for the environment, and often small enough to run locally on edge compute such as your mobile or PC without the need for an internet connection,” says Silvia Lehnis, consulting director for data and AI at UBDS Digital.

The environmental and operational cost benefits make SLMs particularly appealing for businesses aiming to reduce their AI carbon footprint while maintaining data security. “Running the model locally without internet access can also have data privacy advantages, as your data is not being shared with an online application for central logging and monitoring, making it suitable for more sensitive use cases,” adds Lehnis.

It’s a recurring theme. This growing awareness that SLMs can enable a shift away from one-size-fits-all LLMs toward more focused, cost-efficient AI models should change how enterprises think about GenAI use. It could have a broader impact on IT buying, certainly in terms of how CIOs think strategically about what is and isn’t possible with GenAI.

Deloitte’s Tech Trends 2025 report suggests enterprises are now considering SLMs and open source options for the ability to train models on smaller, more accurate datasets. It’s a recognition that size isn’t everything, but accuracy and relevance is, aligning any AI deployments with operational objectives.

The trajectory of AI adoption indicates a growing preference for models that balance performance with operational practicality, but there is also a growing desire for more edge computing, real-time and strategically relevant functionality.

Interestingly, back in 2017, Gartner predicted this would happen, claiming that by this year, 75% of enterprise-generated data would be created and processed outside traditional centralised datacentres or the cloud. And that was before we knew anything about SLMs and their role.

So, what does this mean for the future of SLMs and edge computing devices? Certainly, they will have a significant role to play as enterprises see AI on their terms but also to enable differentiation. That will become the new challenge for CIOs – how to get the best out of GenAI to make a big impact on business performance. Angles for this can come from a number of directions – it really depends on the organisation and the industry.

The rise of SLMs is not just about cost savings or security – it’s about AI differentiation. As Jarrod Vawdrey, field chief data scientist at Domino Data Lab, points out, SLMs are already reshaping healthcare, finance and defence, allowing on-device AI to reduce latency, protect sensitive data and enhance real-time decision-making.

“SLMs deployed on medical devices enable real-time patient monitoring and diagnostic assistance,” he notes, while financial institutions are leveraging SLMs for fraud detection and anti-money laundering compliance.

For CIOs, the challenge is shifting. How do you harness GenAI to make a significant impact on business performance? The answer lies in adapting AI models to industry-specific needs – something SLMs are uniquely positioned to do. The next few years will see enterprises move beyond generic AI models, focusing instead on hyper-relevant, domain-trained AI that drives differentiation and competitive advantage. If anything is going to push edge computing into the mainstream, it’s small language models.