You can use AI on protected health information (PHI) and stay HIPAA compliant. The condition is straightforward: the AI vendor has to be a business associate operating under a signed business associate agreement (BAA) that covers how the tool actually handles your data. Problems start when PHI reaches an AI tool that never signed one, and that happens more often than most healthcare organizations realize.

According to healthcare IT leaders that Paubox surveyed, 85% suspect their staff are using unauthorized AI tools, while only 26% have any visibility into that usage. More than two-thirds had already identified unsanctioned AI adoption inside their organization. Every one of those tools is a place PHI can leave the building without a BAA behind it.

Does HIPAA let you use AI with PHI?

Yes, with conditions. HIPAA never names any specific technology, so there is no rule that says "no AI." What the law regulates is who gets to touch PHI and on what terms.

Under the HIPAA Privacy Rule, any vendor that creates, receives, maintains, or transmits PHI on your behalf is a business associate, and you need a BAA with them before that data changes hands. An AI service that reads a patient message, summarizes a chart, or drafts a clinical note is doing exactly that, so the AI vendor becomes your business associate the moment PHI flows into the tool. The Department of Health and Human Services (HHS) explains the business associate relationship in its guidance for covered entities.

The BAA is the contract that makes the arrangement lawful. It binds the vendor to safeguard the PHI, use it only for the purpose you agreed to, report security incidents, and return or destroy the data when the relationship ends. HHS publishes sample business associate agreement provisions that show what those terms look like. Without a signed BAA, putting PHI into an AI tool is a disclosure to an outside party that HIPAA does not permit.

Where the BAA gap opens up

Most of this risk comes from the free tools people already have open in a browser tab. Consumer AI products generally do not sign BAAs, and many of their terms of service explicitly tell users not to enter health or other sensitive data.

A clinician pastes a patient summary into a general-purpose chatbot to speed up a referral letter. A biller drops an explanation of benefits into an AI tool to reword it. The work gets done faster, and the organization now has PHI sitting with a vendor it has no agreement with.

This is why the shadow AI numbers matter. When 85% of leaders suspect unsanctioned use and only about a quarter can see it, the honest assumption is that PHI is already moving into tools no one vetted. You cannot sign a BAA after the fact for data that already left.

What a BAA has to cover for AI

A BAA on file is only the first step. AI tools have a longer supply chain than a typical vendor, and the agreement has to follow the data all the way down it. Before you trust a tool with PHI, get clear answers to these:

  • Does the BAA cover the exact plan you are using, or only an enterprise tier you have not purchased? Vendors frequently offer a BAA on their business or API product while the consumer version stays out of scope.
  • Does it extend to the model providers and subprocessors behind the tool? An AI application often runs on another company's model or cloud, and the PHI protections have to reach every party in that chain.
  • Does it stop the vendor from using your PHI to train or improve its models? Consumer terms often allow training by default, which is the opposite of what a covered entity needs.
  • How long is PHI retained, and can you require deletion on demand?
  • Where is the data processed and stored, and who inside the vendor can access it?

If the vendor cannot answer these in writing, the BAA is not doing the job you need it to do.

How to run AI on PHI without losing your BAA

Start by getting the BAA signed before a single record flows, and make sure it names the product tier you actually use. A verbal assurance from a sales rep does not count.

Send the tool only the PHI the task requires. HIPAA's minimum necessary standard applies to AI the same as it applies to anything else, so strip the identifiers the model does not need. Where the use case allows it, work with de-identified data instead, using one of the two methods HHS recognizes in its de-identification guidance. Behavioral health and substance use records carry stricter rules under 42 CFR Part 2, so treat those with extra care.

Give staff a sanctioned tool that a BAA already covers. Most shadow AI use comes from people trying to do their jobs faster, not from bad intent, and they will keep reaching for whatever is easiest until a compliant option is just as convenient. Pair that with an inventory of what is already in use so you can close the gaps you can see.

Log and audit AI access to PHI the way you would any other system, and train staff on which tools are approved and why the free ones are off limits. For the full build, see our guide to building with AI in healthcare.

For more information on this topic, watch the on-demand recording from our AI + HIPAA webinar.

AI agents raise the same question

Autonomous AI agents are starting to act on patient data on their own, pulling records and drafting responses without a person pasting anything into a box. The compliance question does not change. An agent that touches PHI is either operating under a BAA or creating an unauthorized disclosure, and "the software did it" is not a defense. As these tools spread, know what the agent can reach and confirm the BAA covers it, then log what it does.

Where Paubox fits

Paubox is a leader in HIPAA compliant email security for healthcare, trusted by more than 8,000 healthcare organizations, and we sign a BAA with every customer. For teams building healthcare applications, the Paubox Email API sends HIPAA compliant transactional email, so the PHI in your notifications and workflows is covered from the start.

If you are still mapping out the basics of keeping patient data safe in transit, our guide to HIPAA compliant email is a good place to start, and you can review the terms of the Paubox business associate agreement directly.

Frequently asked questions

Does ChatGPT sign a BAA?
OpenAI will sign a BAA, but only for specific plans and configurations, and the consumer and lower tiers are usually left out. Terms change often, so confirm the current agreement for the exact plan you intend to use. We keep a running rundown of which providers sign and how to turn it on in how to get a BAA for your AI stack.

Is de-identified data still PHI?
No. Data that has been de-identified under one of the two HHS-recognized methods is no longer PHI, which is why de-identifying inputs is a useful way to shrink your risk when an AI use case does not need real identifiers.

Does a signed BAA make any AI tool safe to use?
No, not alone. A signed BAA makes the use lawful, and you still owe the same safeguards and oversight you apply to every other system that handles PHI. We unpack that gap in why 'HIPAA-ready AI' isn't always HIPAA compliant.