The AI rocketship may be running on fumes

A Long March-5 rocket carrying Chang'e-5 spacecraft blasts off from Wenchang Spacecraft Launch Site on November 24, 2020 in Wenchang, Hainan Province of China.

(Image credit: Wang Longhua/VCG via Getty Images)

Moments after OpenAI, Google, or Anthropic drop a major upgrade to their AI models, you'll see people already speculating on the next update's date and features. And there have been fairly regular updates to feed those rumors. However, those days may be over, according to a Bloomberg report. All three major AI developers are reportedly struggling to make their next-gen models match their ambitions for improvement over the current crop.

The report claims that OpenAI's work on the Orion model isn't going as well as the company expected. The model doesn't perform at the level the company is aiming for, especially when it comes to coding. Orion may not offer a seismic change from GPT-4 compared to how GPT-4 blew GPT-3.5 out of the water. That may be one reason OpenAI CEO Sam Altman publicly pushed back on rumors about the release date for the Orion model and an upgrade to ChatGPT.

All of the AI developers are running into the same ceilings in growing their model's abilities. The biggest is likely training data. The companies have leveraged enormous datasets to train their AI models, but even the internet is not infinite, and that goes even more when discussing high-quality data useful for training AI. Finding previously unused, accessible information is becoming tricky. That's partly because of growing awareness and consideration for ethical and legal rights to use some data, but that's only part of the explanation. At some point, there aren't enough human examples for the AI models to absorb and improve upon. Even if the companies find enough raw data, processing it and incorporating it into an AI model is expensive in terms of money and computing power. If the data cannot make more than slight improvements, then upgrading the AI model might not be worth the price.

Fuel or fumes?

The report describes how OpenAI and its rivals are looking to other ways of upgrading their models, like post-training Orion with human feedback. That's a slow way to improve an AI model and raises questions about whether AI has reached the limits of rapid scaling in size and functions. Brute computing power and avalanches of data may not be enough to make the dreams of AI developers real anymore. They'll need to get more creative in how they iterate on their models without throwing the entire internet at it.

For us, we should expect somewhat slower releases of new and improved AI features. That might not be terrible if it gives everyone a chance to catch their breath and really dig into the best ways to use all the AI tools released in the last few years. There's plenty to explore with ChatGPT-o1. And, who knows, maybe this will give OpenAI the space to work on releasing the Sora AI video creator, which has been kept highly restricted despite OpenAI teasing it with a steady drip of demos.

You might also like...

TOPICS

Eric Hal Schwartz is a freelance writer for TechRadar with more than 15 years of experience covering the intersection of the world and technology. For the last five years, he served as head writer for Voicebot.ai and was on the leading edge of reporting on generative AI and large language models. He's since become an expert on the products of generative AI models, such as OpenAI’s ChatGPT, Anthropic’s Claude, Google Gemini, and every other synthetic media tool. His experience runs the gamut of media, including print, digital, broadcast, and live events. Now, he's continuing to tell the stories people want and need to hear about the rapidly evolving AI space and its impact on their lives. Eric is based in New York City.