Staff can only paste a patient’s email into an AI chatbox if there is a business associate agreement (BAA) with the AI company and the tool is approved for use with patient communications. If your staff pastes into a private or free account that does not have a BAA, they are disclosing protected health information (PHI) that is not allowed under the Privacy Rule. This is true even if the staff member never sends the drafted email.

Paubox surveyed 150 healthcare IT and compliance professionals in the United States. 95% stated that they have staff using AI in their email, and 62% have had staff try to use ChatGPT or another tool that their organization did not approve. When a staff member pastes that information into a tool, it determines which platform will be processing the PHI. When that response is sent back to the patient, you have to ask the same question.

 

Why the AI vendor becomes a business associate

When an outside party creates, receives, maintains, or transmits PHI on behalf of a covered entity, they are considered a business associate under HIPAA. Whether staff are typing into a web page or have an add-on in their email, the AI platform is handling PHI.

Only if your organization has a BAA in place with the AI platform can they receive that disclosure of PHI under 45 CFR 164.502(e). Without a BAA, it would be considered an impermissible disclosure, and the Breach Notification Rule would assume you had a breach unless you could conduct a risk assessment to determine there was a low probability of compromise.

Under federal guidance for cloud computing, a vendor is considered a business associate if it can access your encrypted electronic PHI (ePHI). An AI model will be able to see all of that information. One Therapeutic Advances in Gastroenterology study looked at LLMs in 2026 and discussed how privacy laws "regulate how health information may be shared with external model providers."

 

Built-in email assistants need the same check

If you are using Microsoft 365 or Google Workspace and have the AI assistants in your inbox you may assume that your BAA already covers this. Depending on your agreement with the vendor and what level of service you are paying for, this may or may not be the case.

75% of IT and compliance professionals said that employees may think Microsoft Copilot and other tools are HIPAA compliant. 16% said they did not consult with compliance when allowing staff to use an AI tool for email. Your scheduler would assume that someone checked this when asking their inbox assistant to respond to a healthcare-related question.

In 2024, a JAMA Network study looked at how Stanford Health Care began using AI to help draft responses to patients through their patient portal. They used an AI platform that was HIPAA compliant and had it integrated into their EHR system. If your organization has a list of tools that are approved for staff to use when replying to patients, make sure all AI assistants and accounts are listed on that document. Staff may have personal accounts that are not under your enterprise agreement.

 

What staff would need to strip out first

There are ways to de-identify a message so that it would not fall under HIPAA. There are many identifiers that must be removed under the Safe Harbor method of 45 CFR 164.514(b). This includes identifiers of the patient and their family members, such as name, email address, phone number, and all dates other than the years.

Most patient emails will contain some of this information. Another JAMA Network study in 2023 asked ChatGPT to answer 195 public questions that did not identify the user from Reddit's AskDocs forum. Medical professionals with licenses found that the answers from the chatbot were better than the original question in 78.6% of the 585 questions asked.

Learn more: ChatGPT prompts for HIPAA compliant email

 

How safe AI-drafted clinical replies are

The Lancet Digital Health study done at Brigham and Women's Hospital in Boston in 2023 simulated questions that patients may ask when they have cancer. There were 100 messages that were drafted by GPT-4 and reviewed by 6 Radiation Oncologists. They found that the responses had the potential to cause severe harm in 7.1% of the 156 responses and could cause death in one response. Many of these responses were from incorrectly identifying how urgent a patient's message was.

They state that "if left unedited, could lead to severe harm or death." Since this was a simulated scenario, it may not reflect real-world usage of an AI drafting responses to patients.

When responding to patients through email, you will want to make sure that there is a clinician who reviews any information that may be a symptom. If a patient is asking a question about billing or parking, you may not need to have as much diligence when reviewing a response.

 

What AI drafting does to reply time

In the 2024 JAMA Network study, there was an average of 20% of staff using AI to draft their responses. They found a lower score on task load and exhaustion but did not see a decrease in the time it took to respond to patients or read and write emails. They stated that "there was no improvement in time."

A randomized study looked at 52 primary care physicians at an academic medical system in San Diego. They found that when using an AI to help draft responses, there was an increase in time to read (21.8%) and in the length of the response (17.9%).

 

Sending the finished reply securely

The responses from an AI assistant will be longer than a human may write. During the simulation at Brigham and Women's Hospital, the responses drafted by the AI averaged 169 words compared to 34 words written by physicians who were not assisted by AI.

You want to make sure that when you are sending these longer responses to your patients with potentially more detail about the patient, they are encrypted. The Security Rule says that you should encrypt data during transmission, but this is considered an addressable specification. That means if it is not reasonable and appropriate to encrypt the data, you can implement other safeguards or document why you did not implement encryption under 45 CFR 164.306(d). Patients also have the right to request that you send their information through another method under the right to confidential communications.

Paubox HIPAA compliant email solution will automatically encrypt all outgoing email from your Google Workspace or Microsoft 365 account and deliver it to your patient's inbox. There is no portal or password required for your patients to access their messages. If you use the Premium plan with Paubox Email, you can set up data loss prevention that will look at the content of the message, the recipient, and attachments to make sure it meets your organization's policy.

 

FAQs

Does turning off chat history or model training make a consumer chatbot safe for patient emails?

No, the vendor still receives and processes the PHI in the pasted message, so the disclosure still requires a BAA.

 

Can staff use an AI tool to translate a patient's email into another language?

A translation tool receives PHI the same way a drafting tool does and needs the same BAA, and under Section 1557 of the Affordable Care Act, machine translations of technical patient communications need review by a qualified human translator.

 

What should an organization do if staff already pasted PHI into an unapproved chatbot?

It should treat the paste as a presumed breach, complete the four-factor risk assessment under the Breach Notification Rule, and notify affected patients within 60 days of discovery unless the assessment shows a low probability of compromise.