Researchers found tens of thousands of pages seeded to make chatbots recommend fraudulent contact details for at least 374 companies.

 

What happened

Attackers are filling the web with material designed to be picked up and repeated by AI assistants, so that ChatGPT, Google Gemini, and Google AI Overview present fraudulent phone numbers, email addresses, and login pages as helpful answers, Dark Reading reported. Researchers who identified the campaign, which they call Dark Sourcery, counted tens of thousands of pages carrying fake support numbers, contact addresses, sign-in pages, and software update instructions. At least 374 organizations have been caught up in it, including Fortune 100 companies, airlines, banks, travel companies, and software providers. Attackers focus on customer support, account recovery, refunds, payments, and software downloads, the same reasons patients contact a hospital or health plan.

 

Going deeper

The technique borrows from search engine optimization, the ordinary practice of shaping web content so search engines rank it highly. Attackers apply the same methods so that AI systems retrieve their pages and draw on them when answering questions. Researchers suspect the most effective combination pairs content placed on high-authority domains such as universities and government sites with material on social media, forums, and user review pages, according to Dark Reading. They assess that some models weigh the source, trusting content more when it appears to come from an authoritative organization. The result differs from search poisoning because the false detail appears in the answer rather than in a link the user has to click, and it differs from prompt injection because no one gives the AI an instruction to follow.

 

What was said

The false information "is making the AI answer the user with the false information, without ever following any instruction or prompt by the attacker, essentially bypassing all defenses," researchers told Dark Reading. They described a fraudulent support number being presented as helpful guidance, potentially without the user ever seeing the page that supplied it, and said the campaign remains active with its full effect not yet measurable.

 

In the know

Researchers dialed some of the numbers the chatbots returned. A person answered and offered to help them move a flight or unlock a bank account, then asked for credit card details, Dark Reading reported. They have also recorded complaints from people who handed over payment card information to numbers supplied in chatbot answers. A study published in August and cited in the research found that 91% of people using AI chatbots do not check the answers they receive, which is the behavior the campaign depends on. Their guidance to affected organizations is to treat customer complaints about scams as a signal worth investigating, and to monitor the answers customers actually receive along with the sources those answers cite, comparing returned phone numbers, addresses, and links against verified company records.

 

The big picture

Healthcare brands are already being impersonated in the messages patients receive, with more than 30 health systems issuing warnings this year about emails and texts carrying the MyChart name and logo, and Epic publishing its own guidance for providers and patients, circulated by the American Hospital Association. A chatbot answer reaches the same patient through a channel no provider monitors. Impersonation already accounts for the largest share of reported fraud losses, with $3.5 billion lost to imposter scams during 2025 and close to $1 billion of that going to people posing as businesses, according to the Federal Trade Commission. Organizations can check what assistants currently return for their own billing line, patient portal address, and medical records department, and publish those details prominently enough that a patient comparing them has something authoritative to compare against.

 

FAQs

How does this differ from a poisoned search result?

A search result requires the user to click a link and land on the attacker's page, where they may notice something wrong. Here the fraudulent detail appears inside the assistant's reply as a plain statement, so the user may act on it without visiting any page at all.

 

Why would an AI system trust a university or government website?

Models weigh signals of credibility when deciding what to draw on, and established institutional domains rank highly on those signals. Attackers placing content on such sites, through comment sections, uploaded files, or compromised pages, borrow that standing.

 

What can an organization do about answers it does not control?

Query the assistants directly for its own support, billing, and login details, record what comes back, and repeat the check regularly. Where a fraudulent detail appears, the organization can report it to the platform and to the FTC, and publish correct details in a place patients are directed to.

 

Does this affect AI tools used inside an organization?

It can, where those tools draw on public web content. Assistants retrieving external material can bring the same fraudulent details into internal workflows, which is why verifying phone numbers, links, and software sources before acting on them applies internally as well.

 

What should a patient do with contact details from a chatbot?

Check them against the organization's own website, a billing statement, or an insurance card before calling or entering anything. A number that appears nowhere in the organization's published material is worth treating as unverified regardless of how the assistant presented it.