AI agents are being deployed – but not to full effect

A woman out of focus in the background touches the word AI, lit up in glowing yellow light, in the foreground. The woman is wearing smart glasses — (Image credit: Getty Images)

Deployment has become the wrong measure of progress.

Across every sector, the conversation about AI agents has moved on from whether to deploy them to how quickly more can be added.

Within that shift, a critical assumption has taken hold which now needs re-examining; running agents and getting value from them are not the same thing.

The wrong business case

When AI agents first appeared on enterprise roadmaps, the business plan was almost always built around cost reduction: automating that, reducing headcount here, cut spend there. But this playbook was borrowed from every previous wave of enterprise technology, and for early-stage pilots it was a serviceable framing.

Since then, organizations that have moved beyond pilots into live operations have largely dropped it. The returns they care about now are faster resolution of operational problems and better experience for the people those systems serve. Cost reduction, where it appears, tends to be a byproduct rather than the objective.

A deployment designed to cut costs will be measured on costs. If the same deployment was actually improving resolution speed or reducing failure demand on support teams, that value would go unrecorded and unmade as a case for further investment. The lesson is one as old as time, but one we need to keep reminding ourselves of: get the objective wrong at the start and you can easily make a successful deployment look like a failed one.

Why deployments underperform

A meaningful proportion of AI agent implementations do not meet expectations, and a significant share of organizations have responded by pausing further investment. Before treating this as evidence that the technology doesn’t work, it’s worth looking at what is actually causing this underperformance.

The most common barriers we see are skills gaps, poor business case definition, data quality problems, and the absence of a capable technology partner. Again, none of this is to do with tech problems, but more to do with preparation and execution.

In practice, I see a further problem in that agents need to be perceived as genuinely better than the process they replace by the people doing the work. If engineers and operators don’t feel the benefits, you’re never going to see effective adoption. After that, deployments will fade away before they have the chance to prove themselves. Buy-in, as ever, needs the same attention as the technical implementation.

Defining what success actually looks like

One consequence of deploying agents without agreed success metrics is the inability to demonstrate value even when it is being created. This is a particular problem in IT management, where AI agents are increasingly handling incident detection, triage and resolution.

Mean Time To Resolution (MTTR) is the metric that matters most in this context, and it repays closer examination. The stages of an incident lifecycle are:

Identification,

triage,

isolation,

diagnosis,

fix, and

Verification

Each of these carries a different weight depending on where the current process is slowest. An organization that takes ten minutes to identify an incident but two minutes to resolve it once identified has a different problem than one where diagnosis is more of a constraint. So agents need to be applied to the stage where they will provide a genuine efficiency gain.

Establish the baseline before selecting the intervention and know where time is actually being lost. Then you can set a specific target for reducing it, and measure against that. Without this, it is genuinely difficult to distinguish a successful deployment from a busy one.

The governance gap

Security and governance frameworks are still for the most part built for environments where humans make consequential decisions, even if software executed them. When you introduce autonomous agents into the mix, with the ability to access sensitive data and act on it in real time with limited human oversight, those frameworks become ineffective. This is not a criticism of how they were designed, more a description of a gap that has opened up as deployment has scaled.

When I look at where organizations are most exposed, it tends to be the enterprises whose existing frameworks are too deeply embedded to revisit easily. Legacy architecture is the constraint, and larger organizations carry more of it.

There’s a comparison to be made here with the eras SaaS sprawl and shadow IT. In both cases the technology moved faster than the controls around it, and the cost of establishing those controls retrospectively was higher than building them in would have been. With this in mind it’s easy to see that governance does not act as a brake on deployment of new tech, it’s a pre-requisite that ensures long-term effectiveness.

Integration decisions made late are expensive

Enterprise IT infrastructure is heterogeneous in ways that technology planning tends to underestimate. The mix of public cloud, private hosting and hybrid environments - layered over legacy systems running processes that are poorly documented and harder to change than anyone would prefer - creates conditions that require deliberate architectural thinking from the start. Agents designed without accounting for this environment will require significant rework once they encounter it.

There is also a less obvious use for AI in this process. Applied earlier in the planning cycle, it can identify where legacy systems are creating the most friction and where integration investment will produce the most return. Most organizations deploy AI to generate output; fewer use it to improve the quality of the decisions that shape deployments in the first place, making this application of agents a competitive differentiator.

The sequencing question

Fundamentally, the technology used in successful AI agent deployments and failed ones is the same. What separates them is sequencing: the conditions for success were established before the agents went live.

Those conditions require more discipline than sophistication, including tightly-scoped use cases, clean, well-governed data, integration as a priority and security frameworks that account for the presence of autonomous systems.

The question worth sitting with is whether your organization knows, specifically, what each AI agent is supposed to improve, whether it is improving it, and what will happen to that agent in eighteen months if it is not. Most enterprises cannot answer all three – if you can, you’ll already be one step ahead of the curve.

We list the best IT Automation software.

This article was produced as part of TechRadar Pro Perspectives, our channel to feature the best and brightest minds in the technology industry today.

The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/pro/perspectives-how-to-submit

TOPICS

Chief Technology Officer (CTO) at KTSL.