Flaw in Google Gemini allows email summaries to be used for phishing

Researchers have shown that attackers can manipulate Gemini-generated email summaries to deliver hidden phishing prompts.

What happened

A researcher has demonstrated how attackers can exploit Google Gemini for Workspace to insert malicious instructions into email summaries. The flaw involves indirect prompt injection, where hidden commands embedded in the email body are interpreted by Gemini when summarizing the message. These summaries can appear trustworthy but may include fake security alerts or instructions leading users to phishing schemes.

Although Google has implemented safeguards against prompt injection since 2024, this specific technique still bypasses defenses, making it a potential tool for phishing without the use of obvious links or attachments.

Going deeper

Marco Figueroa, Mozilla’s GenAI Bug Bounty Programs Manager, reported the flaw through 0din, Mozilla’s bug bounty platform. The attack method uses HTML and CSS to hide a prompt in an email's body by setting font size to zero and color to white, making the directive invisible in Gmail but readable to Gemini.

Once the recipient asks Gemini to summarize the email, the hidden prompt is executed. In Figueroa’s example, Gemini falsely warns the user that their Gmail password has been compromised and includes a fake support phone number. Since many users see Gemini summaries as an extension of Google Workspace, such alerts may be perceived as legitimate.

Figueroa recommends filtering or neutralizing hidden text before it reaches Gemini and scanning summaries for urgent messages, links, or phone numbers as part of a post-processing review.

What was said

A Google spokesperson responded to the disclosure by noting that the company is actively strengthening its defenses against adversarial prompts through red-teaming and additional layers of mitigation. While some safeguards are already in place, others are being rolled out. Google also said it has found no evidence so far of real-world exploitation using the method demonstrated.

FAQs

What is prompt injection in AI systems?

Prompt injection is when hidden or manipulated inputs are used to influence how an AI model responds, often tricking it into producing misleading or malicious output.

Why are hidden prompts hard to detect in emails?

Attackers can use HTML and CSS to make text invisible in email clients, so while users don’t see the instructions, AI models like Gemini can still parse and act on them.

How can organizations detect or prevent these attacks?

Security teams can implement filters to detect hidden styling in emails and review AI-generated outputs for red flags like urgency, phone numbers, or unsupported claims.

Does this vulnerability require users to click links or download attachments?

No. The attack works without traditional phishing elements. It relies on users requesting an AI summary, which then delivers the malicious message.

Can Gemini summaries be trusted for security-related alerts?

No. Gemini summaries should not be treated as authoritative sources for security alerts. Users should verify messages through official support channels or IT departments.