HIPAA does not apply to machine learning models operations and functioning. However, it does cover how protected health information (PHI) is used to create a model. The vendors and employees working with it and any messages created from the model's findings. So long as you can establish an allowed purpose for the PHI, confirm it is still PHI or has been de-identified, and ensure proper agreements cover everyone working with the data or output, you are compliant.

Under HHS rules, a business associate is permitted to use or disclose PHI only if their business associate agreement (BAA) authorizes it or the law mandates it. Many times this is done over email. It could be a dataset shared over email, a message sent to someone regarding an output, or an output that generates a message. In Paubox's 2026 Healthcare Email Security Report, we found 170 email-related breaches in 2025; 74% of these breached domains were not using DMARC effectively.

 

Does training a model fit a permitted purpose?

Yes, provided it falls under a permitted use of PHI. Covered entities should aim to apply it toward health care operations. Quality assessment and improvement, evaluating outcomes and population-based activities to improve health or lower cost, are some examples. One thing to watch out for is making sure your analyses don't yield generalizable knowledge. When the knowledge you're seeking about your patients could extend beyond your own practice, it may fall under research. Research will require different measures to ensure you're following HIPAA such as getting authorization from your patient or obtaining an IRB/privacy board waiver.

If you run a clinic and want to forecast no-shows using your past scheduling data, using PHI could be permissible. However, if your goal is to train a model you intend to sell to others, you're moving beyond that purpose. Research from Paubox on shadow AI revealed that 25% of respondents lacked any formal approval for staff using AI in email. Privacy and AI article author Blake Murdoch, having reviewed the law, notes that healthcare AI "can be manageable for the purposes of protecting privacy." However, there are some caveats.

 

Which agreement covers a vendor building the model?

Anyone that you share PHI with to help you create a model or store your model should have a BAA in place. Be sure the contract permits them to use your data to develop a product for their customers. Another option is to permit them to de-identify the data and incorporate it into their product. OCR allows for business associates to de-identify PHI if their agreement allows for it.

Say you're a telehealth company that provides medication management and you want to share your patients' visit records with a company that can help you predict when they might run out of medication. Have that conversation and put an agreement in place first before you start sharing your data with them. According to a study in the Journal of Medical Internet Research: Medical Informatics, 6,612 breaches were reported to OCR from 2009 to 2025. There were 1,950 breaches (29.5%) involving business associates and 44.8% of individuals affected by a breach involved a business associate. Paubox's 2026 figures revealed that vendors and business associates were involved in 28% of email-related breaches. You can read more in our statistics.

Read also: When does AI become a business associate under HIPAA?

 

What de-identification settles and what it leaves open

When data is de-identified, it falls outside the definition of PHI. There are two ways to de-identify data under HIPAA. Safe Harbor removes 18 listed identifiers and requires no actual knowledge that the remainder could identify someone, while Expert Determination requires a qualified expert to document a very small risk of re-identification. Both methods still carry a risk of re-identifying an individual according to OCR.

The data analyzed in this study came from the National Health and Nutrition Examination Survey. They applied a random forest algorithm to re-identify demographic data and 20-minute activity data down to record numbers. They were able to identify 94.9% of adults and 87.4% of children in the 2003-2004 dataset. Concluding that "current practices for deidentifying physical activity data are insufficient for privacy." Keep in mind this data was collected from wearable technology and not medical records. When sending any data to your data scientists treat it as PHI until you've completed a Safe Harbor review or hired an expert to de-identify it. This goes for email as well.

Learn more: Can de-identified data be used to train AI under HIPAA?

 

What a trained model can reveal about its training data

You can test to see if a model knows about certain records through membership inference. The researchers discovered that de-identification of real clinical notes doesn't safeguard records from membership inference attacks. Since these researchers were testing the attack, there's no way to know how often this could happen on hospital models.

Since HIPAA doesn't go into detail on models, you'll want to include this in your risk analysis. Where you store your trained models, fine-tuning data sets, and any outputs that include record-level data. These should all be sent over encrypted channels just like your data. Including when your data scientist sends over email to a vendor.

 

How model outputs reach patients by email

Whether you're sending someone a risk score, reminding them to take their medications, or explaining lab results to a patient. These are all messages that contain PHI. Under the Security Rule you can send these to someone via email. Encryption is only an addressable specification, so if an individual asks for their information via email, you can send it to them. After giving them a short warning about the risk of sending it via email. HHS will only allow this when someone asks for their information. So you'll still be liable for any impermissible disclosures if you're sending information to others. If you create a model that reminds your patient to take their medication, you should be sending those securely.

There was a quality improvement project at Stanford Health Care that had clinicians use a model to help them respond to patient messages. Over the course of 5 weeks, 162 clinicians participated, and the mean utilization rate of drafts was 20%. They were given the option to use a model or not when responding to patients. And only one health system was studied.

Paubox surveyed 170 IT leaders in healthcare and asked them how mature their organization was when it comes to email security. Our lowest scoring category was encryption/recipient experience with a score of 2.39/4. 48% of respondents said they require their recipient to login to a portal to read an encrypted message. Paubox's Email API will allow you to send encrypted messages from your application. We also offer a business associate agreement.

 

Which email safeguards the model pipeline needs

Your risk analysis should cover all areas where electronic PHI is stored. Which means you should look at where your extracts, notebooks, and models live. An employee could get phished and give access to your training data. In a study published on JAMA Network Open, employees clicked on 422,062 phishing emails out of 2,971,945. Between the years 2011 and 2018 across six health care institutions in the United States.

Paubox's 2026 report showed that 53% of email-related breaches happened on Microsoft 365 in 2025. That's up from 43% in 2024. Our research on organizational maturity found that 64% have experienced a confirmed or suspected AI-generated email attack. Only 38% have AI based threat detection in place. You can implement inbound email security and authentication to help decrease this risk.

 

What compliance cannot establish about a model

HIPAA can't prove that your model is accurate or will produce fair results. An external validation of the Epic Sepsis Model was conducted at Michigan Medicine. They found an AUROC of .63 and did not detect sepsis in 67% of patients with sepsis. They also alerted on 18% of hospitalizations. Describing their results as "poor discrimination and calibration in predicting the onset of sepsis." While this was only one health system, it's important to validate your model.

There are many other regulations you'll want to follow as well. Making sure you aren't discriminating and following state laws. Some states require you to let a patient know you're using AI to diagnose or treat them. Our shadow AI research also showed that 75% of leaders think their employees know that tools like Microsoft Copilot are HIPAA compliant. With the new proposed security rule changes, we may have to start encrypting all data. Until then, we recommend you document why you choose to encrypt your data.

See also: HIPAA Compliant Email: The Definitive Guide (2026 Update)

 

FAQs

Can patients access risk scores generated by a machine learning model?

Generally, yes, when those scores form part of their designated record set, the records maintained by or for a covered entity include information used to make decisions about individuals.

 

Can a model process PHI on servers outside the United States?

HIPAA does not prohibit overseas processing or storage. Organizations must still meet applicable HIPAA requirements, including required business associate agreements.

 

Can patients refuse to have their PHI used for model training?

Patients can request restrictions on uses for health care operations, but covered entities generally do not have to agree.