What technology leaders need to ensure AI delivers

Artificial intelligence is often pitched as an “existential” issue for enterprises.

But for all their undoubted enthusiasm for and investment in AI tools, it seems many technology leaders are failing to treat it as an enterprise app.

Evidence is already piling up that many AI projects are failing to make it beyond the experimentation or pilot stage.

Over 40 percent of agentic AI projects will be canceled by the end of 2027, Gartner predicts, often because of inadequate risk controls and uncertain ROI.

This failure to launch wastes investment, and corrodes confidence in the technology in the longer term.

This creates a gap between those organizations which are steadily making the transition to enterprise AI and those that are struggling to make it work. The gap will only open further as Gen AI gives way to Agentic AI.

Having a vision is key to AI success

Of course, having a vision is key to AI success, as is having the data to inform your enterprises’ distinctive AI strategy. This, together with some seed investment, might be enough to deliver a dazzling pilot.

But is that enough to ensure success at enterprise scale? Gartner’s figures show, clearly, no.

So what is missing? What is it that technology leaders must do to ensure that AI doesn’t just delight, but actually delivers?

The answer is to ensure operational readiness for AI. Put simply, this is the ability to deploy, manage and scale AI out of the labs and across the entire organization.

That means putting in the hard work of ensuring that what starts as a compelling but disconnected pilot is woven into the enterprise at large.

It means ensuring AI runs across a unified platform that spans compute, data and governance. A platform that can be replicated throughout the organization, whether on prem, in the cloud or at the edge.

There’s nothing new about the basic concept. Rolling out any business critical workload such ERP or CRM successfully demands the same focus on underlying operational infrastructure.

That said, there are specific challenges to highlight when it comes to achieving this with AI.

Setting up AI infrastructure

It’s easy to think that the AI infrastructure management begins and ends with GPUs. But high bandwidth memory, fast storage, and networking to match all play their part. As do other processors and accelerators, depending on which part of the workflow we’re looking at.

Most importantly, that infrastructure – whether on-prem, in the cloud, or hybrid – needs to be able to adapt and scale as projects move from local pilot to enterprise production. AI, by its nature, may be much more sticky than more traditional corporate workloads.

But this is more than a question of processor horsepower or gigabytes of storage. Security and governance is non-negotiable when it comes to enterprise AI projects. The underlying data and an organization's own models are key to their future, and must be held close.

Data sovereignty and AI regulations more broadly further complicate matters. Tech leaders need to know their data is where they say it is, and to be clear exactly who can – and cannot – access it.

The possibilities of AI are limitless. But so is the price tag if this underlying infrastructure is not managed appropriately. Just paying for GPUs, and the power to run them, then leaving them underutilized blows a hole in ROI as well as undermining ESG commitments.

Operation scale out

Technology leaders need to plan how they scale capacity up – and down – from the outset. But they also need to be able to manage and predict costs. So, they need confidence that their platform and tool kit allows them to do this easily.

This becomes even more critical as AI agents come into the picture. Security, governance and compliance needs to be ensured even as agents access and generate data and make decisions.

Infrastructure must be able to support them and handle spikes in demand as their actions play out. The location of assets must be considered to reduce latency for inference workloads running in real time. And energy use must be kept within acceptable bounds.

Once all of this is taken into account, the outline of what operational readiness in the AI age becomes clearer.

True operational readiness demands a turnkey approach to AI, in the shape of a full stack platform, with the ability to span GPUs and the other accelerators that are needed.

It must include integrated data services, supporting the full range of formats AI will need, together with security and governance controls to match.

And it should support both VMs and containers, with the ability to orchestrate these. Racing towards operationalizing AI is challenging enough. No-one wants to be running a cloud native migration at the same time.

The role of LLMs

LLMs might not always deliver repeatable answers. But the infrastructure Gen AI and Agentic AI relies on must be repeatable if companies are to scale it in line with demand.

That includes the cloud, as well as on-prem and the edge.

When they have the right platform and tools, tech leaders can ensure their staff are focused on steadily maximizing the value they can gain from their AI investments.

Not spending time and resources trying to transform a successful pilot project into an enterprise wide strategy.

Whether betting the farm on AI or recognizing that AI will be part of their broader tool kit, technology leaders have to recognize AI is an enterprise app.

And enterprise apps need enterprise grade infrastructure that can support them from the pilot stage, into production and into the future.

Because that’s what will assure their organisation’s existence in the long term.

We list the best data visualization tools.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro