AI-enabled impersonation is making fraud more believable

AI-enabled impersonation uses advanced AI (especially deep learning) to mimic real people’s voices, faces, and writing style. In practice, attackers use deepfakes, AI-generated audio/video, to create synthetic media that impersonates executives, clinicians, or vendors. One Cognitive Research: Principles and Implications study defines deepfakes as “synthetic media created by deep-generative methods to fake a person’s audio-visual representation."

For example, AI can clone a person’s voice from as little as 30 seconds of recording, enabling a fraudster to mimic a CEO on a call. The result is scams that appear to come from trusted insiders.

What is AI-enabled impersonation?

Recent studies show that AI-generated fakes can deceive even vigilant humans. Voice cloning and video synthesis have improved so much that people often cannot tell the difference. For instance, the above-mentioned deepfake study found that human observers classified deepfake faces no better than chance. The study specifically notes, “Good accuracy in feature classification. Humans, in contrast, experienced challenges in distinguishing between real and deepfake static images. Their classification accuracy was at chance level, and this underperformance relative to machines was accompanied by a truth bias and low confidence for the detection of deepfake images. Using dynamic video stimuli, Study 2 found that performance…”

Likewise, the journal article Mitigating the harms of manipulated media: Confronting deepfakes and digital deception notes that “audio deepfakes continue their ballistic trajectory in terms of realism, ease of use, and accessibility.”

Modern deepfakes capture accent, emotion, and facial cues so well that even colleagues may believe the imposter. Criminals have already exploited this: Farid cites real cases where an AI-synthesized voice duped staff into sending large wire transfers. Such realism, an urgent request spoken in a familiar voice, greatly boosts the scam’s credibility.

AI impersonation vs traditional phishing

AI-enabled impersonation extends classic phishing by adding voice, video, and highly personalized content. Traditional phishing or business email compromise (BEC) relies on spoofed email or malicious links. In contrast, deepfake scams may arrive via live phone or video call with a known person, bypassing email filters. For example, the journal article describes a deepfake video conference in which attackers posed as the CFO and tricked a finance team into wiring $25 million.

The article warns, “The trend of the past few years has been that all forms of image, video, and audio deepfakes continue their ballistic trajectory in terms of realism, ease of use, and accessibility. Generative AI is well on its way to passing through the uncanny valley.”

The attacks also use public data to script believable dialogue (the tone of a CEO or doctor). In effect, AI impersonation combines real-time social engineering (voice/video) with phishing goals. The multimodal approach is novel: it gives scammers realistic human cues that standard email-based defences were never designed to catch.

How to verify suspicious requests

Given these new threats, organizations must verify any unexpected requests through independent channels. Security experts, including healthcare vendors like Paubox, advise treating unusual emails, calls, or texts as suspect. A simple control is a callback: if someone (even a boss) requests a wire or private data, staff should call the sender at a known number to confirm. Security experts, including healthcare vendors like Paubox, advise treating unusual emails, calls, or texts as suspect because people are not reliable deepfake detectors.

In the deepfake study, it was found that humans classified static real and deepfake face images with only 49% overall accuracy, and they misclassified deepfake images as real 69% of the time. Video detection was better but still imperfect, with humans reaching 63% overall accuracy, while one machine-learning video detector performed at only 49% accuracy. A simple control is a callback: if someone, even a boss, requests a wire transfer or private data, staff should call the sender at a known number to confirm.

Implementing multi-person approval for large transactions and strict payment controls is also recommended. Training must emphasize that hearing a familiar voice or seeing a familiar face is no proof of identity, especially when research cited in the article found that people mistook an AI-generated voice for the real person 80% of the time.

Technical safeguards, including email authentication and encryption, complement these steps. Although formal studies on verification protocols for deepfakes are limited, the evidence that people and detection tools can be fooled supports the need for out-of-band confirmation, stronger payment policies, and layered impersonation defenses, which aligns with the practical logic behind Paubox’s ExecProtect solution.

Implementing multi-person approval for large transactions and strict payment controls is also recommended. Training must emphasize that hearing a familiar voice is no proof. Technical safeguards (e.g., email authentication and encryption) complement these steps. Although formal studies on verification protocols for deepfakes are limited, the evidence that people are easily fooled requires out-of-band confirmation and robust policies, as Paubox’s ExecProtect solution also advocates in practice.

See also: HIPAA Compliant Email: The Definitive Guide (2026 Update)

FAQs

How are deepfakes different from cheapfakes?

Deepfakes use AI-generated or AI-manipulated content, while cheapfakes use simpler editing tricks, such as slowing audio, cropping video, or taking real footage out of context. Both can mislead people, but deepfakes are harder to spot because they can imitate realistic human features.

Why is voice cloning especially risky?

Voice cloning is risky because people often treat a familiar voice as proof of identity.

Can deepfake detection tools solve the problem?

Detection tools help, but they are not a complete solution.