For years, the standard advice for spotting a phishing email was to look for bad grammar, awkward phrasing, and generic greetings. That advice is no longer accurate, because with the advent of AI, a well-written email can be just as dangerous. A phishing email in 2026 can be written in flawless English, reference a real project the recipient is working on, match the tone of a colleague, and arrive at exactly the moment it is most likely to be believed. Criminals now have their own AI platforms, and a growing underground economy built around bending mainstream ones to their purposes.

 

Where the malicious platforms came from

The public release of ChatGPT in late 2022 changed attacker behavior within about a year. Legitimate large language models, the AI systems trained to generate human-like text, came with safety guardrails designed to refuse harmful requests. Criminals wanted the same capability without the refusals, and a market appeared to supply it.

The first widely known tool was WormGPT, built on an open-source language model and marketed on dark web forums specifically for writing business email compromise messages, a type of attack where a criminal impersonates an executive or vendor to trick staff into transferring money or data. FraudGPT followed shortly after, advertised as an all-in-one criminal toolkit that could generate phishing pages, write malware, and identify vulnerabilities, sold on a subscription model that mirrored legitimate software pricing. Others came quickly behind them, and the FBI issued a formal public service announcement warning that criminals were exploiting generative AI to commit fraud at greater scale and believability than before.

What makes this generation of tools different is how closely they now resemble legitimate products. Google's Threat Intelligence Group documented multiple criminal AI platforms operating through 2025 with pricing tiers, customer support channels, and upgrade paths, packaged and sold the way any commercial software subscription would be.

Read more: What is phishing? | What is business email compromise?

 

Stealing access to the real thing

Not every attacker bothers building a separate criminal model. Many simply steal access to the mainstream ones, which are more capable anyway.

Microsoft's Digital Crimes Unit tracked a global network it designated Storm-2139 that exploited stolen API keys to get into commercial AI services, including Azure OpenAI, and bypass the safety controls those services enforce. An API key is basically a password that lets one piece of software talk to another, and when a criminal steals a healthcare organization's or a developer's key, they can run a frontier AI model on someone else's account and someone else's bill, with the usage buried inside legitimate traffic. Microsoft disrupted the network through a civil complaint filed in December 2024, followed by criminal referrals to the FBI and international partners in early 2025.

The significance for a healthcare organization is that its own leaked credentials can become the engine for attacks against others, and that the criminals producing convincing phishing at scale are often doing it with the same models used for legitimate work, stripped of the guardrails.

 

Why this defeats the old defenses

Speed is the first change; IBM researchers demonstrated in 2023 that AI could construct a functional phishing campaign in about five minutes using five prompts, a task that took a team of human security experts roughly 16 hours to match. That gap means an attacker can now produce in an afternoon what used to take a week, and can generate hundreds of unique variations of a single message rather than reusing one template.

The variation breaks a core assumption behind traditional email filtering. Signature-based detection, the approach that flags messages matching known-bad patterns, depends on attacks looking similar to each other. When every message in a campaign has a different subject line, sender name, and body, there is no consistent signature to match against. The Verizon 2026 Data Breach Investigations Report documented that attackers are now using generative AI across a widening range of attack techniques, augmenting known methods rather than inventing new ones, which makes those familiar attacks faster and harder to catch at the same time.

Personalization is the third shift, and for healthcare it may be the most dangerous. AI tools can scrape a target's public information, like LinkedIn profiles, hospital staff directories, or published research papers, and write a message that references real details convincingly. A phishing email that mentions a genuine upcoming conference, names a real colleague, and matches the writing style of the department it claims to come from does not trigger the instinct that a clumsy generic message would.

 

Why healthcare is a natural target

Healthcare staff work under conditions that make AI-generated phishing especially effective. Clinical and administrative staff move fast, handle a high volume of email, and routinely receive legitimate messages about referrals, insurance claims, lab results, and vendor invoices. A well-crafted AI phishing message that imitates any of those does not stand out. The IBM experiment above made the point directly, because the test that pitted AI-generated phishing against human-crafted messages was run on the staff of a healthcare organization, and two other healthcare organizations that originally agreed to participate withdrew before testing began because they expected the click rates to be too high.

The FBI's 2025 Internet Crime Report recorded business email compromise as the second-costliest cybercrime category in the country, with reported losses exceeding $3 billion, and healthcare organizations are frequent targets because they process large payments to many vendors and business associates. An AI-generated message impersonating a known vendor and requesting an updated payment account is exactly the kind of attack these tools now produce at scale.

The entry problem compounds it. Paubox's 2025 Healthcare Email Security Report found that only 5% of known phishing attacks in healthcare get reported by employees to security teams. When the messages were easy to spot, that low reporting rate was already a problem. Now that AI has removed the obvious tells, expecting staff to catch what a trained eye would once have flagged is no longer a realistic defense on its own.

 

What actually works against AI-generated phishing

The defenses that still function share one trait. They look at behavior and context rather than the content of the message, because content is exactly what AI has learned to fake.

Paubox's 2026 Healthcare Email Security Report tracked a 47% increase in attacks avoiding native email defenses in 2025, which confirms that the default filtering built into Microsoft 365 and Google Workspace is not keeping pace with AI-generated campaigns. Paubox Inbound Email Security uses AI to analyze sender behavior, message intent, and contextual signals rather than scanning for known-bad content, which is the approach that holds up when the text itself looks legitimate. Fighting AI-generated attacks effectively means using AI on the defensive side too, trained to catch the behavioral anomalies that a perfectly written message cannot hide.

Sender authentication adds a layer that AI cannot easily forge. Standards like DMARC, DKIM, and SPF, which are protocols that verify a message actually came from the domain it claims to come from, catch impersonation attempts regardless of how convincing the message body reads. An AI can write a flawless email pretending to be a vendor, but it cannot make that email pass an authentication check it has no legitimate access to.

Learn more: Protect your email from AI-generated attacks

 

In the news

Coverage through 2025 and into 2026 has tracked AI-generated phishing moving from a minority of attacks to a dominant share of what reaches inboxes. TechTimes reported in July 2026 that AI-assisted attacks had surged to roughly four in ten of all reported phishing attempts, driven by criminals using generative AI to mass-produce convincing lures at almost no cost or skill. The same reporting noted the expansion of AI beyond email into voice cloning and text-based scams, part of a broader pattern where the same technology gets adapted across every channel an attacker can reach. The consistent takeaway across the coverage is that the barrier to producing convincing attacks has collapsed, and defenses built around spotting human error no longer match the threat.

 

FAQs

Is it illegal to access the dark web?

No. The dark web itself is just a hidden part of the internet reached through anonymizing software like Tor, and it has legitimate uses for privacy, journalism, and circumventing censorship. Buying or selling stolen data on it is illegal, but visiting the network is not a crime in itself.

 

What is WormGPT and FraudGPT?

They are AI tools built or modified specifically for criminal use, marketed on dark web forums for writing phishing emails, creating malware, and running business email compromise campaigns without the safety restrictions that mainstream AI models enforce. They are sold on subscription models that resemble legitimate software, which has made sophisticated attack capability available to criminals with little technical skill.

 

How is AI-generated phishing different from regular phishing?

The main differences are quality, speed, and personalization. AI produces messages with perfect grammar and natural tone, generates hundreds of unique variations in minutes rather than hours, and can tailor each message using a target's public information. The old advice to watch for typos and awkward phrasing no longer works because those tells are gone.

 

Can attackers really use legitimate AI tools like ChatGPT for attacks?

Yes, in two ways. Some manipulate mainstream tools into producing harmful content despite their safeguards, and others steal API keys, the credentials that grant software access to a service, to run frontier AI models on someone else's account while bypassing the provider's controls. Microsoft's Digital Crimes Unit disrupted one such global network in 2025.

 

Why is healthcare especially exposed to AI phishing?

Healthcare staff handle high email volumes under time pressure and routinely receive legitimate messages about claims, referrals, and vendor payments, which AI-generated phishing can convincingly imitate. Healthcare also processes large vendor payments, making it a frequent target for business email compromise, and only 5% of phishing attempts in the sector are reported by employees.

 

How can an organization defend against phishing that has no obvious tells?

Effective defenses analyze behavior and context rather than message content, since content is what AI fakes best. AI-powered inbound filtering that evaluates sender behavior and intent, combined with sender authentication standards that verify a message's true origin, catch attacks that look completely legitimate on the surface.