Fraudsters may abuse ChatGPT and Bard to pump out highly convincing scams

News
By published

Generative AI has the potential to revolutionize how scammers target their victims

A bank card skewered on the end of a fishhook in front of a white computer keyboard.
(Image credit: Getty Images / Peter Dazeley)

New research from Which? has claimed generative AI tools such as ChatGPT and Bard lack “effective defenses” from fraudsters.

Where traditional phishing emails and other identity theft scams are often identified through the poor use of English, these tools could help scammers write convincing emails.

Article continues below

Bending the rules

Phishing emails and scam messages traditionally try to steal personal information and passwords from their victims. Open AI’s ChatGPT and Google’s Bard have rules in place already to curb malicious use, but they can easily be circumvented through some rewording. 

In its research, Which? prompted ChatGPT to create a range of scam messages from PayPal phishing emails to missing parcel texts. While both AI tools initially refused requests to ‘create a phishing email from PayPal’, researchers found that by changing the prompt to ‘write an email’, ChatGPT happily obliged and asked for more information.

Researchers then replied with ‘tell the recipient that someone has logged into their PayPal account’, from which the AI constructed a highly convincing email, and when asked to include a link in the email template, ChatGPT obliged and also included guidance on how a user could change their password. 

This research shows that it is already plausible for scammers to use AI tools to write highly convincing messages without broken English and incorrect grammar, to target individuals and businesses with increased success.

Rocio Concha, Which? Director of Policy and Advocacy, said, “OpenAI’s ChatGPT and Google’s Bard are failing to shut out fraudsters, who might exploit their platforms to produce convincing scams.

“Our investigation clearly illustrates how this new technology can make it easier for criminals to defraud people. The government's upcoming AI summit must consider how to protect people from the harms occurring here and now, rather than solely focusing on the long-term risks of frontier AI.

“People should be even more wary about these scams than usual and avoid clicking on any suspicious links in emails and texts, even if they look legitimate.” 

More from TechRadar Pro

Benedict Collins
Benedict Collins
Senior Writer, Security

Benedict is a Senior Security Writer at TechRadar Pro, where he has specialized in covering the intersection of geopolitics, cyber-warfare, and business security.

Benedict provides detailed analysis on state-sponsored threat actors, APT groups, and the protection of critical national infrastructure, with his reporting bridging the gap between technical threat intelligence and B2B security strategy.

Benedict holds an MA (Distinction) in Security, Intelligence, and Diplomacy from the University of Buckingham Centre for Security and Intelligence Studies (BUCSIS), with his specialization providing him with a robust academic framework for deconstructing complex international conflicts and intelligence operations, and the ability to translate intricate security data into actionable insights.