Can AI agents collect PHI safely?

An artificial intelligence (AI) agent refers to a system that can take instructions, process information, and perform tasks with some degree of autonomy. It could be a chatbot that answers questions or a scheduling assistant that uses data to coordinate care. Many AI agents used by covered entities have to ingest sensitive patient information to provide these personalized recommendations. Generative models, according to a study by researchers Chen et al. have the ability to go about “transforming medical diagnostics, research, treatment planning, and patient care.”The risk factor comes in when we consider that a lot of the data these agents use is in the category of protected health information (PHI).

AI’s appetite for data also comes with serious privacy risks. Reidentification, which infers a patient’s identity from supposedly anonymized data, ranks as one of the most insidious threats. In the Research Handbook on Health, AI and the Law, it is warned that AI “undermines the already weak power of de-identification to protect health privacy by making it easier to re-identify patients, either individually or at scale.”

For underresourced health systems, AI agents and their many functions can democratize expert knowledge and deliver care to remote areas via chatbots and telemedicine. Used responsibly, these capabilities have the potential to improve efficiency, reduce clinician burnout, and expand access to quality care, but without consideration for HIPAA compliance, the risk becomes far greater than the reward.

A look at email security in the news

Email remains one of the most common sources of HIPAA-reportable breaches. Paubox’s 2026 Healthcare Email Security Report analyzed 170 email-related healthcare breaches reported to HHS in 2025. According to the report, those breaches exposed PHI for 2.5 million individuals. The report also found that 60% of surveyed healthcare IT leaders rated their email security as inadequate, while 72% said their infrastructure needed a major overhaul. Only 23% said they were confident their email security was fully effective.

The survey findings show that healthcare IT leaders understand the gap. Paubox reported that 89% of healthcare IT leaders identified AI and machine learning as critical for email threat detection, but only 44% had deployed AI-powered email security tools. Healthcare IT leaders also estimated that employees report only 5% of known phishing attacks to their security teams.

HIPAA compliant email forms create a safer front door for AI collection

An AI agent should never be allowed to become a potential exposure point for PHI. The safer model begins with a HIPAA compliant email form since the form defines what the patient submits, where the data goes, who receives it, and how the organization documents the exchange. The form is the control point between the patient and the AI system.

A paper on privacy protection in healthcare AI by Tuan Pham warns that “AI may threaten personal privacy. AI systems can find meaning in tiny details. A combination of a symptom, appointment type, medication name, ZIP code, and free text note can generate a very sensitive profile. A HIPAA compliant form limits unnecessary exposure because it can restrict intake to the minimum fields necessary for the particular purpose.

Secure routing keeps AI useful without removing human responsibility

A Hoonaker et al. study on secure messaging in primary care found, “Secure messaging has the potential to improve communication and information flow.” The same logic applies to HIPAA compliant email forms. As soon as a patient completes an intake form, the submission reaches the care team through a secure channel. The AI agent can then identify missing fields, categorize the request, detect urgency indicators, or prepare a draft response for staff review. The workflow becomes faster without making the AI the final decision-maker.

A safe AI agent should know when human intervention is required in order to continue the workflow, a characteristic that separates secure agents from the alternative. HIPAA compliant forms help because each form can carry purpose, routing logic, access rules, and audit metadata from the start without the need for developer input from the client's end.

A form submission can trigger a confirmation to the patient, an internal alert to the correct team, and a documented record of delivery. The AI agent can support the process, but the secure email system records the movement of PHI. The record matters for compliance reviews, incident response, and internal accountability.

FAQs

What is privacy preserving AI?

Privacy preserving AI refers to AI systems designed to reduce exposure of sensitive data while still allowing the system to perform useful tasks.

Does using deidentified data remove AI privacy risk?

Deidentification lowers risk, but it does not remove it.

What is data minimization in privacy-preserving AI?

Data minimization means only giving the AI system the information it needs to perform a specific task.