Behind the curtain: how IT pros can spot AI washing

Person using a phone and laptop in a cafe.
(Image credit: Shutterstock)

The benefits of generative AI have been made abundantly clear in recent years, with businesses set to reap valuable productivity and efficiency gains, not to mention an uptick in staff morale Gen AI can offer by reducing their time spent doing simple tasks. Productivity and automation tools are benefitting hugely from this AI supercharge, but sadly the lines between the companies offering genuine AI solutions and those simply marketing themselves as such are becoming increasingly blurred.

As a result, we’re seeing businesses often struggling to separate hype from reality when looking for the right AI tools. IT leaders are increasingly being burdened with spotting the difference between true AI tools with integrity and imposters, as digital transformations that are successful are now dependent on this ability to distinguish fact from marketing slogans. It's no longer enough for tech vendors to just market products as “AI-driven” – IT pros are looking for evidence to back it up.

This year, I expect many businesses will be better equipped to sort the diamonds from the coal, and shallow AI solutions will find themselves exposed as the latter.

Prasad Ramakrishnan

CIO, Freshworks.

An eye for AI

Fortunately, IT professionals haven’t been deterred from implementing AI - there are too many real benefits gained from utilizing the right tools. In fact, Freshworks found that a very significant 71% of IT workers currently rely on AI to support their workloads. The survey found that the Gen AI boom is directly influencing IT teams to review whether they are using AI in their workflows, and that they’ve become more informed and less easily swayed by extravagant claims of AI’s capabilities that don’t have proven examples to back them up. The rise of overhyped AI solutions - similarly to the rise of cloud computing - has led to a collective awareness of key indicators as to the validity of AI tools’ claims.

IT practitioners are now rightfully demanding solid evidence of the effectiveness of AI solutions, and this emphasis on accountability around AI has driven teams to exercise caution in streamlining their technology stacks. There’s a real emphasis on ROI for IT teams, leading them to make sure the tools they’re shopping for aren’t just improving productivity, but are also delivering provable value to the business.

This has provided a perfect environment for genuinely useful AI tool providers to make a name for themselves in the IT space, as CIOs are on the lookout for software that can integrate AI into workflows with real impact. As time passes, the reputation of those contributing to genuine productivity gains will rise, making it easier to identify the providers with the appropriate solutions for specific needs.

Selecting the right solution

So what differences should businesses be on the lookout for?

Let’s take customer support chatbots as an example. Depending on complexity, a range of chatbot products are available; some more capable than others of conversational support. The best AI chatbots will harness natural language processing and machine learning technologies to provide conversational and contextual support, cutting resolution times and freeing up support agent time for cases that require more of a human touch. 

Simpler rule-based algorithmic chatbots certainly also have their value - provided they’re being used appropriately for basic automation of basic customer queries. But if a business is looking for a state-of-the-art AI chatbot that provides meaningful customer interaction, adapting tone and behavior according to customer needs, then only a chatbot with the right capabilities will work.

An issue arises when there is a mismatch between the customer’s needs and the solution they’ve selected – and with recent marketing sometimes applying the AI sticker a bit too generously, IT teams should be clearly defining their requirements and checking their chosen solution ticks all the boxes. To avoid disappointment, organizations with a real need for AI-informed customer interactions need to rigorously administer testing practices to ensure they’re paying for the product they expect. Although different products are useful in their respective use cases, no-one wants to pay for one thing and receive another.

AI, clear my schedule

The right AI assistant can provide significant business benefits, and the insights AI copilots can deliver are already being utilized in a variety of industries. For customer support agents for example, virtual AI assistants are helping prioritize cases and summarizing customer issues, leading to increased agent efficiency and higher customer satisfaction.

Looking closer at some AI virtual assistants exposes further basic rule-based algorithms and pre-defined templates however. This can manifest in the form of schedule management – customers purchasing AI assistants expect the ability to understand natural language commands and organize a schedule without actively directing themselves.

Real AI technology will be able to understand complex scheduling preferences, taking on responsibility to make decisions without user input, and will remain adaptable to shifting circumstances. Assistants without proper AI capabilities will fall into the trap of waiting on specific keywords and phrases to perform simple scheduling actions, offering little in terms of real productivity gains.

IT teams should be testing any potential virtual assistant tools for actual AI use by observing how they respond to natural conversation, handling more complex tasks and how its behavior changes over time. Adaptation is a keystone of genuine AI, and if a virtual assistant is consistent in its failures - even when corrected – it might just be falsely marketed technology.

Testing our patience, or patiently testing?

There’s a theme here – test, test, and test again. Wider implementation may take a little longer, but doing the work upfront can prevent future awkward conversations with the CIO when the question of return on AI investment is brought about.

IT teams should be engaging in constant app rationalization; evaluating AI tools against clear parameters for success can lead to more consistent productivity enhancements. The right SaaS solution for any company should be one that can demonstrably provide value without unnecessary complexity, and the faster that IT practitioners can discern between AI-washing and genuine AI tools, the faster their technology stack can be futureproofed.

We've listed the best virtual assistant services.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro

Prasad Ramakrishnan

Prasad Ramakrishnan, CIO, Freshworks.