I am an AI expert and here's what businesses should know about using popular AI chatbots for writing content

The AI hype felt relentless in 2023/24. While the initial frenzy has subsided somewhat, executives and professionals now grapple with the reality of deploying Artificial Intelligence (AI), specifically Generative AI (GenAI), within their organization.

LLMs (Large Language Models), the technology behind popular GenAI chatbots, are powerful, but there remains a significant disconnect between the perception of what they can do and their practical application for business writing.

Easy to use interfaces like ChatGPT make GenAI seem like it "can literally do anything".

This is a dangerous misconception. While incredibly useful for certain tasks, GenAI chatbots can be totally useless, and even harmful when not used appropriately.

Fundamental differences

The fundamental difference lies in how GenAI works compared to traditional software.

1. Traditional software is deterministic

It follows fixed logic and algorithms, producing the exact same, 100% accurate, and therefore repeatable result every time you give it the same input. Think of hitting CTRL+F in Word – you get a precise, repeatable count of a term.

2. Generative AI is non-deterministic

LLMs predict the next word based on probabilities from their training data. This means asking the same question twice will often give you different answers. They are designed to be variable.

Critical characteristics to understand

This core difference results in two critical characteristics businesses must understand:

1. Hallucinations: GenAI can confidently generate incorrect information or make things up. This isn't a bug; it's how the technology works. It's guessing based on patterns, not verifying facts. Copilot, for example, can wildly miscalculate readability scores or miss most instances of a search term.

2. Lack of Repeatability: You simply cannot guarantee the same output from the same prompt.

Here is the absolute critical takeaway: if your writing or document review task requires 100% accuracy or 100% repeatability, you must use deterministic software, not GenAI. Using GenAI for tasks demanding precision is a classic case of wielding a "GenAI hammer" and seeing every problem as a nail.

Flaws and errors in practise

Consider the disastrous consequences. I’ve used MS Copilot to search for every instance of "cybersecurity" in a contract for compliance purposes, only for the GenAI tool to miss 23 out of 27 occurrences. Trying to "shred" a document line-by-line into an Excel matrix for compliance, a task requiring perfect repeatability, is another inappropriate use case where GenAI will fail.

For businesses, especially in regulated sectors, using GenAI for tasks where factual accuracy is paramount is dangerous. Users may trust outputs due to brand credibility, not realizing the risks of inaccuracy.

Real-world failures like Air Canada's chatbot providing false information resulting in a lawsuit underscore the significant brand and trust damage inaccurate GenAI can cause.

So, where IS GenAI useful for business writing?

GenAI thrives for tasks where variability, creativity, or a "good enough" answer is acceptable or desired.

Appropriate use cases include:

• First Draft Creation: Generating initial versions of documents like management plans, executive summaries, or proposal sections based on context. This can save significant time.
• Creative Assistance: Rewriting content in a different tone or style.
• Summarization: Condensing lengthy documents.
• Simplification/Rephrasing: Making complex text more accessible or refining paragraphs.
• Research & Analysis: Using public data for competitive analysis or sales research where perfect accuracy on every detail isn't required for generating insights. Using NLP (another type of AI) for thematic analysis across communications to check message consistency.

Beyond simple chatbots, the real value often lies in specialized applications. These layer GenAI into workflows for specific jobs, intelligently combining GenAI for creative/drafting tasks with deterministic software for accuracy-critical functions like readability scoring or compliance checks.

They understand the "job to be done" and apply the right technology. NotebookLM, which generates audio summaries of documents, is a great example of a focused application.

Garbage In, Garbage Out: The Unsexy Truth of Knowledge Management

Generative AI, even when combined with techniques like Retrieval Augmented Generation (RAG) to access proprietary data, is not a magic wand that can overcome poor data quality. The old adage "garbage in, garbage out" is more relevant than ever. If your internal knowledge bases are a mess of outdated content, multiple revisions, and poorly tagged documents, the AI's output will reflect that chaos.

As the Harvard Business Review noted, "Companies need to address data integration and mastering before attempting to access data with generative AI". Good data hygiene – clear folder structures, naming conventions, and processes for maintaining content – is crucial but is fundamentally a human behavior problem, not just a tech one. Investing in proper knowledge management now will pay dividends when you roll out any GenAI solution.

Data Security: The Enterprise Achilles' Heel

Many popular AI chatbots rely on public cloud-based LLMs. For businesses, especially those in regulated industries like defense, finance, and healthcare, feeding proprietary or sensitive or PII (Personally Identifiable Information) data into these public models poses a significant security risk. CISOs (Chief Information Security Officers) are rightly wary, often blocking interactions with such models entirely.

The safer path for enterprises involves hosting LLMs in a private cloud or on-premise, fully locked down behind the firewall. The rise of powerful open-source models like Llama 4 or Mistral Nemo which can be deployed securely in-house, is a welcome trend. This shift is so significant that a Barclays CIO survey last year indicated 83% plan to repatriate some workloads from the public cloud, largely driven by AI considerations.

The Real Driver: People and Process

Most AI projects fail not due to the technology, but because of people, process, security, and data issues. Lack of buy-in, poor strategy, inadequate data, and insufficient change management and user education are common pitfalls.

Deploying AI chatbots without teaching users about:

• Hallucinations
• The need to verify outputs
• Effective prompting
• Crucially, what tasks not to use GenAI for

...will lead to frustration and project failure.

Start with the business problem you need to solve, then map the appropriate technology to that job. Don't just chase the "shiny new tech". Define your goals, measure success (both quantitative and qualitative), and involve end-users early.

When evaluating vendors, look beyond captivating demos. Ask pointed questions about accuracy, repeatability, data handling, security posture, and their understanding of your specific use cases and industry needs. Always try before you buy and vet vendors carefully. Be wary of vendors who overpromise or claim GenAI can do everything.

In summary, popular AI chatbots offer exciting capabilities, but they are not magic. They are powerful tools with significant limitations. Successful businesses will adopt a pragmatic, thoughtful approach: understanding GenAI's non-deterministic nature, applying it strategically to appropriate tasks (like creative drafting), leveraging hybrid applications, investing in data quality and security, and crucially, focusing on the people and processes required for effective adoption and change management.

This is the path to truly unlocking AI's value.

I tried 70+ best AI tools.

This article was produced as part of TechRadarPro's Expert Insights channel where we feature the best and brightest minds in the technology industry today. The views expressed here are those of the author and are not necessarily those of TechRadarPro or Future plc. If you are interested in contributing find out more here: https://www.techradar.com/news/submit-your-story-to-techradar-pro