Google Gemini can be hijacked to display fake email summaries in phishing scams

News
By published

What if Gemini told you your Gmail account was compromised?

Gmail at 20
(Image credit: Getty Images)
  • Gemini in Workspace presents unique opportunities for fraud, researchers warn
  • The AI tool can be tricked to display fake security warnings
  • Businesses should make sure invisible text is not processed by the AI

Cybercriminals have found a creative new way to abuse Google’s Generative Artificial Intelligence (GenAI) to steal people’s Gmail accounts.

Google introduced Gemini, its AI-powered chatbot assistant into its Workspace suite of productivity apps some time ago, and one of the things Gemini can do is summarize incoming emails - so when a person receives an email, they can bring up a vertical pane on the right-hand side of the screen, asking Gemini for assistance with different things, such as bringing up vital email information, adding calendar entries, and more.

However experts have warned this also opens up the Gmail accounts for so-called “prompt-injection” attacks - so if the incoming email message contains a hidden prompt for Gemini, it can be executed in the pane.

Is Gemini phishing for your password?

According to security researcher Marco Figueroa, this is exactly what the email provider is now susceptible to.

By using HTML and CSS, threat actors can add a prompt for Gemini, with its font size set to zero, and its color to white. Therefore, the victim will not be able to see it, but Gemini will act on it. If that prompt makes Gemini display a phishing message, it will do just that, and since the message would come from a trusted source, it increases the chances of success.

Figueroa showed how a malicious prompt could notify the victim that their email account has been compromised, and that they need to “call” Google on a phone number displayed in the message to resolve the issue.

To protect against future prompt injection attacks, companies should make sure their email clients remove, neutralize, or ignore content that is styled to be hidden in the body text. Furthermore, they could include a post-processing filter that scans the inbox for “urgent messages”, URLs, or phone numbers.

Finally, businesses should educate their employees that summaries provided by the Gemini tool should not be a replacement for security alerts.

Via BleepingComputer

You might also like

TOPICS
Sead Fadilpašić

Sead is a seasoned freelance journalist based in Sarajevo, Bosnia and Herzegovina. He writes about IT (cloud, IoT, 5G, VPN) and cybersecurity (ransomware, data breaches, laws and regulations). In his career, spanning more than a decade, he’s written for numerous media outlets, including Al Jazeera Balkans. He’s also held several modules on content writing for Represent Communications.

You must confirm your public display name before commenting

Please logout and then login again, you will then be prompted to enter your display name.