A two-year tracking study of AI adoption on criminal forums documents the shift from AI as a writing aid to AI as core operational infrastructure across six distinct attack functions.

 

What happened

Researchers tracking AI adoption in cybercriminal operations over two years have documented a shift from AI being used primarily to polish phishing email text to AI being embedded as core infrastructure across multiple stages of an attack. According to Infosecurity Magazine, the analysis reviewed incidents across threat actors ranging from ShinyHunters to North Korean state-linked groups, finding that AI consistently appeared in two roles, embedded in the attack workflow itself, and used as a lure exploiting user trust in AI brands. In 2024, AI use was largely limited to improving phishing email fluency and generating basic scripts alongside tools like FraudGPT. Around mid-2025, the picture had expanded to include deepfake services and a growing underground market for AI-enabled tools. Around mid-2026, researchers found AI involved in phishing page generation, credential harvester development, code obfuscation to defeat static analysis tools, social engineering content, identity fabrication for fraud schemes, and malicious installations disguised as AI brand downloads.

 

Going deeper

Researchers identified six documented functions AI now serves in active intrusions. At the top of the list is phishing at scale, where AI generates unique phishing pages and lures rapidly enough to launch, adjust, and repeat campaigns continuously.

The second is faster malicious tool production, where AI generates web shell code that attackers plant on compromised servers to maintain access and credential harvesters, while varying or padding code to frustrate the security tools that detect known malicious patterns.

Third is social engineering polish, where AI eliminates the grammar errors, awkward phrasing, and clumsy design that awareness training teaches staff to spot.

Fourth is identity fabrication, where AI accelerates the creation of fake professional profiles and deepfakes used in North Korean IT worker fraud schemes, making it harder for hiring managers to detect fraudulent candidates in interviews.

Fifth is initial access acceleration, where AI-generated obfuscation speeds up ClickFix attacks and AI-assisted pages support device code phishing campaigns.

And sixth is AI-branded lures, where attackers disguise malicious installation commands or browser extensions as legitimate AI tools, including Claude downloads, to trick users into running them.

 

What was said

Researchers stated in their analysis cited by Infosecurity Magazine that AI "consistently enabled these operators to achieve more, faster, with less effort," and that threat actors treat AI "as operational infrastructure, something to buy, tune and slot into existing workflows," with the primary focus on "balancing efficiency with reliability and cost." On defense, researchers concluded that "security teams don't need a new strategy built around AI as a category. But AI does change the pace of attacks, so they do need strong fundamentals, defense-in-depth, and AI and automation wherever operationally possible to match the new pace."

 

In the know

The North Korean IT worker fraud dimension carries specific healthcare implications. The FBI has previously warned that North Korean operatives have targeted healthcare organizations specifically, seeking remote employment to gain insider access to systems and data. According to BleepingComputer, the FBI has identified hundreds of US companies infiltrated through this scheme, with healthcare among the targeted sectors. AI-generated fake profiles and deepfake video capabilities documented in the analysis directly accelerate that threat by making fraudulent candidates harder to screen out during video interviews, the stage where hiring teams most commonly expect to verify identity.

 

The big picture

The practical implication for healthcare is not that AI has invented new attack categories but that it has removed the friction that previously limited how fast and how broadly existing attacks could scale. A phishing campaign that previously required hours of manual lure crafting per target can now generate thousands of unique personalized lures in minutes. A credential harvester that previously required coding skills to build can now be generated and varied by an AI agent. According to Paubox's Shadow AI report, 95% of healthcare organizations report staff using unapproved AI tools, and 75% of healthcare workers incorrectly assume Microsoft Copilot is automatically HIPAA compliant. The same AI tools that healthcare staff are adopting without oversight are being monitored and weaponized by the criminal ecosystem against them.

 

FAQs

What is a web shell, and why does AI-assisted generation make it more dangerous?

A web shell is a malicious script that attackers upload to a compromised server, giving them ongoing remote access through a web browser. Previously, building an effective web shell required coding knowledge. AI can now generate functional web shells on demand and vary their code structure to avoid matching the signatures that security tools use to detect known versions.

 

What is North Korean IT worker fraud, and how does AI make it harder to detect?

North Korean operatives apply for remote employment at foreign companies using fabricated identities, then use that insider access to steal data, plant malware, or funnel payments to fund state programs. AI generates convincing fake professional profiles, portfolio work, and deepfake videos for interviews, removing the inconsistencies that previously helped hiring teams identify fraudulent candidates.

 

Why does AI-generated code padding frustrate security tools?

Security tools that detect malware using static analysis look for known malicious code patterns or signatures. When AI varies or pads malicious code, adding irrelevant but syntactically valid content between functional sections, each version looks structurally different from the last, preventing signature-based detection from matching the new variant against known threats.

 

What does "AI as the lure" mean in practice for healthcare staff?

Attackers disguise malicious files or browser extensions as downloads of legitimate AI tools, exploiting the trust users place in familiar brand names like Claude, ChatGPT, or Copilot. A healthcare worker who installs what appears to be an AI productivity tool may be installing a credential harvester or remote access tool instead, with no visual indication that the download is fraudulent.

 

What defensive changes does the shift to AI-paced attacks require?

Behavioral detection that monitors what processes actually do rather than what they look like addresses AI-varied malware that defeats signature matching. Automated containment that can isolate a compromised account or endpoint within seconds addresses the compressed time between compromise and lateral movement. Out-of-band verification for sensitive requests requiring a phone call or in-person confirmation, rather than an email approval, addresses AI-polished social engineering that makes fraudulent requests indistinguishable from legitimate ones.