Small models, big wins: four reasons enterprises are choosing SLMs over LLMs

In recent years, Large Language Models (LLMs) have dominated mainstream attention for their ability to generate human-like responses, and complete tasks ranging from summarizing documents to coding applications. But now, enterprises are beginning to question if these general-purpose AI models can actually support their specific business goals.

A recent McKinsey survey found almost three quarters of organizations regularly use generative AI in at least one business function, up from 65% in early 2024. Despite this rising popularity, there is still a need for greater control and understanding of how AI models work. Nearly two-thirds (62%) of business leaders cite model explainability and data management among common challenges with AI adoption. If organizations are to keep pace and bring their ambitions for AI to fruition, they need to act quickly to overcome these issues.

Small solutions for large problems

Purpose-built Small Language Models (SLMs) can offer significant advantages in enabling AI to drive more targeted use cases. SLMs are designed for depth, not breadth – unlike LLMs that aim to ‘know everything’. This means they can deliver more relevant, cost-effective and accurate outcomes when trained on focused datasets that are tailored to specific business needs.

Here are four key reasons why enterprises are opting for SLMs over LLMs:

1. Fewer parameters, more efficient functions

Although smaller in size, the more focused nature of SLMs can make them more effective. There are two main forms - either open-source models like TinyLlama and Phi-2, or propriety models that are fine-tuned for specific industries such as law, healthcare or finance.

In finance, for example, by training SLMs exclusively on regulatory filings, internal data and earnings transcripts, they can assist with valuable tasks such as risk assessment and financial reporting. Meanwhile in healthcare, SLMs can assist doctors with diagnostics based on insights from medical journals and clinical records, using terminology and recommendations that are relevant to real-world practice and regulatory compliance.

By prioritizing quality over quantity, SLMs can equip enterprises with a toolkit that accounts for all industry-specific niches far better than any generalist LLM model.

2. Accuracy through context

Almost all companies are investing in AI, yet only 1% believe their models have reached a mature state. AI models often lack the accuracy, scalability or reliability needed to make them viable for use in business applications, preventing organizations from generating real value from their investments. To overcome this, AI models need to be trained more effectively, with richer context that enables them to understand the challenges they have been built to solve.

Because of their general-purpose capabilities, LLMs often get things wrong. For instance, we’ve seen lawyers mistakenly using ChatGPT to cite non-existent judicial cases, and being sanctioned after using false information provided by the AI model. The need to thoroughly validate AI-generated outputs from LLMs to ensure reliability and accuracy adds a step to legal workflows, rather than speeding things up.

SLMs, on the other hand, significantly reduce the risk of this type of hallucination because they are trained on organizational or domain-specific data. By remaining grounded in relevant context, SLMs are more likely to improve accuracy and avoid ambiguity.

Reducing the risk of misleading outputs is particularly crucial for highly regulated industries, like finance. For instance, an LLM could think ‘AP refers to ‘Accounts Payable’, when analysing a networking whitepaper about ‘Access Points’ – leading to confusing or inaccurate outputs.

In addition, SLMs dramatically improve the experience for the end user. Employees will be able to interact with the AI using the terminology they’re used to, without spending excessive time ‘translating’ their intent, or watching out for hallucinations.

3. Reduced cost and infrastructure needs

Facilitating state-of-the-art LLMs requires significant IT infrastructure and is often incredibly costly. This cost remains a critical concern to enterprises, with recent reports revealing that 96.9% of business leaders say high computational costs are a major limitation for their organization's ability to leverage AI and ML successfully. For enterprises that require repeatable, domain-specific outputs from AI, these costs associated with LLMs simply aren’t worth it.

Meanwhile, many SLMs can run on consumer-grade hardware like NVIDIA Jetson, Mac Mini, or AI-enabled laptops which can cost under $1,000. This enables enterprises to invest in AI to suit their needs, without paying excessive prices.

A recent example is NVIDIA’s ‘Spark’ – the world’s smallest AI supercomputer, capable of running models with up to 200 billion parameters. Innovations like Spark open the door to high-performance AI without incurring huge costs.

4. Improved privacy, control and customization

Finally, one of the greatest concerns with LLMs is their ‘black box’ nature. Enterprises have no control over the amount or details of the open web datasets LLMs have been trained on, and in turn, it is difficult to guarantee their sensitive data is not being shared or used by third parties. SLMs allow enterprises to overcome this risk by maintaining control of the data that is available to their AI.

This enables organizations to fine-tune AI models in-house, creating a ‘knowledge moat’ of unique data that competitors don’t have access to. In addition, SLMs can allow organizations to build in compliance with frameworks like GDPR more effectively and reduce the risk of bias or errors – which is especially crucial for businesses in regulated industries.

Taking the hybrid approach

SLMs aren’t going to replace LLMs: they both work for different scenarios. Going forward, hybrid AI ecosystems with agentic frameworks will determine which model to use based on context, sensitivity and user intent.

This rise of modular AI stacks, combining edge-based SLMs, secure data layers and cloud-scale LLMs will unlock a new era of enterprise intelligence. As all industries mature their approach to AI, it’s important to remember that precision matters just as much as power.

We list the best Large Language Models (LLMs) for coding.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro