Gemini security flaw raises phishing fears

A newly identified vulnerability affecting Google’s Gemini artificial intelligence system has heightened concerns about the exposure of Gmail users to sophisticated phishing and account compromise, sharpening the debate around how large language models interpret and act on hidden instructions embedded in everyday digital content.

Security researchers and policy analysts say the issue stems from indirect prompt injection, a technique that allows malicious instructions to be concealed inside emails, documents or web pages in ways that are invisible to human readers but legible to AI systems. When an AI assistant processes such content, it can be manipulated into taking unintended actions, including generating deceptive responses, extracting sensitive information or assisting attackers in social-engineering campaigns.

The Centre for Emerging Technology and Security at The Alan Turing Institute has described indirect prompt injection as generative AI’s most serious security weakness. The centre has warned that language models do not parse information as humans do, making it possible to insert instructions that appear benign on the surface yet fundamentally alter an AI system’s behaviour. Because modern AI assistants can ingest content from emails, attachments and external web pages, the potential attack surface is both wide and difficult to monitor.

In the context of Gmail, analysts say the risk lies in the growing use of AI tools to summarise emails, draft replies or flag priority messages. A carefully crafted phishing email could include hidden commands designed to influence Gemini’s output, nudging users towards unsafe actions or generating responses that appear trustworthy but direct them to malicious links or fraudulent payment requests. While the attack does not automatically grant access to an account, it could materially increase the success rate of phishing campaigns by exploiting trust in AI-generated guidance.

Researchers within Google have publicly acknowledged the scale of the problem. Teams at Google DeepMind have outlined methods for continuously detecting indirect prompt injection attempts, focusing on identifying anomalous patterns in model behaviour rather than relying solely on static filters. The approach reflects a recognition that attackers adapt quickly and that defences must evolve in tandem.

Google has also described a layered mitigation strategy aimed at reducing the impact of prompt injection attacks across its AI products. This includes stricter content sanitisation, separation between untrusted input and system instructions, and improved monitoring to flag suspicious interactions. The company has emphasised that no single control is sufficient and that resilience depends on multiple safeguards operating together.

Despite these measures, independent experts caution that structural challenges remain. Large language models are designed to be flexible and context-aware, qualities that make them valuable to users but also attractive targets for manipulation. Unlike traditional software vulnerabilities, prompt injection exploits the interpretive nature of AI, blurring the line between data and instruction. That ambiguity complicates efforts to apply conventional security models.

The issue has implications beyond Gmail. As AI assistants are increasingly integrated into productivity suites, customer service platforms and enterprise workflows, indirect prompt injection could be used to influence automated decision-making, leak proprietary information or undermine compliance processes. Academic studies have shown that even simple hidden prompts can override safety constraints under certain conditions, raising questions about how reliably models can distinguish between legitimate user intent and adversarial input.

Industry observers note that awareness of the threat has grown sharply over the past year, with regulators and standards bodies beginning to examine AI-specific security risks. Some enterprises have responded by limiting the types of data that AI tools can access or by requiring human review for AI-assisted actions involving sensitive information. Others are investing in specialised security tooling designed to audit and constrain model behaviour.