Become a TechRadar Insider
CNN's Wolf Blitzer leveled his inquisitive gaze at me and asked the biggest question in tech, “Is AI an existential threat to humanity?" I smiled confidently and assured Wolf, "No, I don't think it is."
But just days after that TV interview, we have, it seems, ample evidence that one of the industry's leaders and core AI developers, Anthropic, perhaps thinks otherwise, and is now urging not just caution, but a slowdown.
Anthropic is suggesting that all AI developers, policy makers, and others join hands and agree to pause or slow down frontier model development so that we can get ahead of the dangers of recursive development.
In a lengthy blog post, Anthropic executives explain that misalignment between the needs and desires of humans and whatever compels AI to improve itself could result in us losing control of them.
It's a scary thought, and one I hadn't entirely considered during my short interview with CNN. I mean, I get the concern now held by both everyday people and Anthropic. In the chat, I tried to explain that disruption and danger can feel similar, especially when the former is moving so fast that it can feel reckless, or at least beyond our comprehension.
I have been arguing for tech and AI regulation for years. It seems like something almost everyone agrees on, but no one can figure out how to broadly implement it. Instead, we get piecemeal bits from local municipalities and Executive Orders. Slowdowns or pauses are not necessarily equivalent to regulation, and what Anthropic is suggesting is something different.
It wants some sort of global agreement across everyone working on frontier models to create a mechanism for delimiting AI development, and maybe even a large red 'Stop all work' button that we can hit when signs of imminent danger to humans arise.
Any rational person would break out laughing at this point. I'm not chuckling at the intention, but at the sheer impossibility of the ask.
I'd explain why this won't work, but Anthropic does a fair job of it in the very same blog post:
A meaningful slowdown or pause would require multiple well-resourced labs at or near the frontier, in multiple countries, agreeing to stop under the same conditions. It would also require that each can verify that the others have actually stopped. Due to the unique characteristics of AI systems, the detectability... element of this arms control problem is much more challenging than with other technologies. Training runs are far easier to conceal than missile silos, their inputs are general-purpose, and the incentive to defect quietly is enormous, because whoever continues while others pause could inherit the lead. A credible pause also has to specify what triggers it, what lifts it, and who adjudicates.
Anthropic notes that while nuclear non-proliferation was easier to monitor around the globe because, well, it's hard to hide when you're building missile silos, AI frontier model development can be hidden on inscrutable systems where no one knows how you're training and testing the models, let alone the output. If we switch the the resursive delopment model, then AI model building is now a black box. Anyone could stand outside the AI dev center and confidently say they "have no knowledge of AI development." Which does not mean it's not happening, just that they have no input on its course.
Also, what if Anthropic somehow got Meta, OpenAI, Perplexity, Microsoft, and others to agree to pausing some work? China certainly wouldn't agree, and then our greatest fears are realized: The US falls way behind while China delivers world-altering models — and we never catch up.
Anthropic knows this, and yet it's suggesting this global agreement. Some believe it's doing it so that others will pause while it continues, or that its current apparent lead with powerful models like Mythos might be preserved.
I don't believe that. I just think they're all standing there, watching Claude build its latest models and wondering, "What have we done?"
It's a level of introspection that might be missing from, say, OpenAI, Meta, and others, but that doesn't necessarily serve a purpose.
Instead of trying to get everyone to agree to take a beat, how about we all agree that humans are always in the loop? Sure, it'll be self policed, but the more we inject ourselves into the development and dissemination of the models, the more we ensure that future frontier models do not rise to the level of existential threat.
Still, I wonder if I need to have another chat with Wolf.
Follow TechRadar on Google News and add us as a preferred source to get our expert news, reviews, and opinion in your feeds.
